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Technical Assessment Report 
1.0 Notification and Authorization 

The NASA Engineering and Safety Center (NESC) received a request from the NASA Associate 
Administrator (AA) for Human Exploration and Operations Mission Directorate (HEOMD), to 
quantitatively evaluate the individual performance of three light detection and ranging (LIDAR) 
rendezvous sensors flown as orbiter’s development test objective (DTO) on Space 
Transportation System (STS)-127, STS-133, STS-134, and STS-135. 

The Assessment Plan was approved by the NESC Review Board (NRB) on March 8, 2012. The 
preliminary stakeholder summary was approved by the NRB on January 3 1, 2013. Mr. Neil 
Dennehy, NASA Technical Fellow for Guidance, Navigation, and Control (GN&C) at the 
Goddard Space Flight Center (GSFC), was selected to lead this assessment. 

The primary stakeholders for this assessment were Mr. William Gerstenmaier, AA for HEOMD, 
Orion Multi-Purpose Crew Vehicle (MPCV) Project Manager; Ms. Kathy Lueders, International 
Space Station Visiting Vehicle Office; and Mr. Steve Labbe, Chief of NASA's Applied 
Aerospace and Flight Mechanics Division. 

The secondary stakeholders for this assessment were Mr. Fra nk Cepollini, Spacecraft Servicing 
Program Office; Mr. Rich Mrozinski, Automated Rendezvous and Docking Community of 
Practice; and Dr. Ken Lebsock, GN&C Technical Discipline Team. 
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4.0 Executive Summary 

Light detection and ranging (LIDAR) sensors perform critical rendezvous relative navigation 
(RelNav) sensing from initial long-range acquisition at several kilometers (km) and tracking of 
the target spacecraft/body through terminal range/bearing measurements for proximity 
operations necessary for docking/berthing/capture. This assessment quantitatively evaluated the 
performance of three LIDAR rendezvous sensors: the Sensor Test for Orion RelNav Risk 
Mitigation (STORRM) Vision Navigation Sensor (VNS), DragonEye, and Triangulation and 
LIDAR (TriDAR). These sensors were flown as orb iter development test objective (DTO’s) on 
Space Transportation System (STS)-127, STS-133, STS-134, and STS-135 from 2009-2011. All 
three systems were developmental units and not qualified flight systems. 

The assessment team performed an independent, statistically-based analysis of LIDAR sensor 
data sets generated from the DTOs. Each LIDAR sensor’s DTO performance is summarized 
relative to their individual DTO performance specifications, when provided, in range and bearing 
and areas of anomalous/unexplained data are identified. The assessment does not address future 
mission and vehicle specific objectives and specifications. In addition, detailed investigations of 
instrument and flight anomalies were out of scope for this assessment. 

For performance analysis, each sensor’s estimated range and bearing was compared to the best 
estimated trajectory (BET) computed from the trajectory control sensor (TCS) measurements 
through the Rendezvous and Proximity Operations Program (RPOP). The residual difference 
between the sensor and BET was statistically summarized, incorporating modeling of systematic 
biases and accounting for time series correlated data structure, as required. 

The STORRM DTO met its objectives and provided operational and performance data to 
continue maturing the system design. All data processing and centroiding were performed post- 
flight, rather than on-board. Range performance was out of specification for a single range bin 
(162-324 meters). A 2-meter bias was discovered in ground testing and confirmed from 
comparison to the BET and pose-based estimates of range. The team found that the noise 
analysis method of second-differencing provided in the STORRM report [ref. 9] led to 
underestimates of noise performance in some cases when autocorrelation was present in the data. 
It is recommended that the STORRM VNS processing development be conducted to mature the 
algorithms that will be implemented within the sensor and navigation system to provide in-flight, 
on-board range, and bearing. 

The DragonEye DTO met its qualitative objectives. However, performance specifications for the 
DTO were not established. The DragonEye bearing measurements did not meet the Dragon 
vehicle specifications during tracking of a single reflector. The team found that intensity peak 
determination impacts range measurement and pixel saturation impacts ability to identify the 
actual peak. Saturation impacts centroiding and feature identification. The DragonEye 
performance is adversely affected by host vehicle rotation with respect to the local vertical, local 
horizontal (LVLH) reference frame and when the host vehicle undergoes translational 


NESC Request No.: TI- 11-00753 



NASA Engineering and Safety Center 
Technical Assessment Report 

Document #: 

NESC-RP- 

11-00753 

Version: 

1.0 

Title: 

Relative Navigation Rendezvous Sensor DTO 
Performance Evaluation 

Page #: 

12 of 103 


maneuvers. There are significant differences in the maturity of the systems flown on STS- 127 
and STS-133, and the team focused its analysis on STS-133. It is recommended to consider 
adjusting the camera design to achieve the specifications, and/or the DragonEye specifications 
should be changed to reflect flight requirements. 

The TriDAR DTO met its qualitative objectives. However, performance specifications for the 
DTO were not established. TriDAR data processing was performed on-board, real-time during 
the DTO. The team found a 1 -meter bias in the range measurements from 200-meters to dock. 

In long-range blob mode, TriDAR erroneously identified a single solar array instead of the 
International Space Station (ISS), which resulted in range errors of 20 meters and bearing errors 
of 12 degrees. Several data processing and recording anomalies were found in the data including 
asynchronous output of pose parameters, time-based stale measurement data that were flagged as 
valid. It is recommended that the root cause of the bias be determined, which may involve 
reconciliation of computer-aided drawing (CAD) geometry, reference coordinate systems, and/or 
more extensive ground testing that could help determine the range error. It is recommended that 
the flight software should include an algorithm to determine if the scan is incomplete based on a 
priori target vehicle knowledge. 

Across all three sensors, obtaining the DTO data sets required approximately 3 months of a 
formal agreement between NASA and the Canadian Space Agency (CSA). It is recommended 
that the Autonomous Rendezvous and Docking (AR&D) Community of Practice (CoP) establish 
data-sharing requirements and agreements prior to conducting joint DTOs. The team found that 
when processing raw sequence or intensity and range data from flash LIDAR sensors, data 
exclusion of obvious erroneous range data due to spurious reflections is critical to measurement 
performance, particularly for real-time flight applications. Furthermore, numerous spurious 
reflections from parts of the ISS, not LIDAR reflectors, occurred for the DragonEye and 
STORRM VNS. By contrast, the TCS, which was a scanning LIDAR used during the Space 
Shuttle Program (SSP) for proximity operations and docking provided few spurious reflections 
throughout its lifetime. These spurious reflections make reflector identification and navigation 
difficult. Overall, it is recommended algorithm development and testing be performed to enable 
robust and reliable preprocessing of flash LIDAR imagery data. Finally, it is recommended 
continued investments be made to the AR&D CoP into algorithm maturity for real-time flight 
applications of LIDAR sensors. 

Ultimately, this assessment improved NASA’s familiarization with and understanding of each 
LIDAR sensors’ hardware, software, and algorithm functionality. By providing insight on 
design and performance of this existing family of LIDARs and their raw data processing RelNav 
algorithms, NASA gained an improved definition of LIDAR sensor performance/functional 
requirements for future sensor procurement specifications. Overall, this assessment improved 
NASA’s smart buyer posture for future LIDAR RelNav sensor procurements. 
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5.0 Assessment Plan 

This assessment quantitatively evaluated the individual performance of three LIDAR rendezvous 
sensors, (i.e., TriDAR, DragonEye, and STORRM VNS), flown as orbiter DTOs on STS- 127, 
STS -133, STS-134, and STS-135. The evaluation was based on the specifications and 
objectives defined by each DTO’s Principal Investigator (PI). 

The primary objectives, as defined by the stakeholders, were: 

1 . Perform an independent, detailed, and statistically-based analysis of LIDAR sensor data 
sets (i.e., raw and processed data) generated from the SSP DTOs. 

2 . Summarize each LIDAR sensor’s DTO performance relative to their individual DTO 
performance specifications. 

3 . Identify specific areas of anomalous/unexplained LIDAR DTO data (for subsequent 
discussion with LIDAR providers). 

Future mission and vehicle-specific objectives and specifications were outside the scope of this 
assessment. Detailed investigations of instrument and flight anomalies were not performed. 
Anomalies, including missing data, were noted, but not investigated. A root cause investigation 
of instrument performance relative to the DTO specification and operational flight anomalies 
were outside the scope of this assessment. However, these anomalies were documented for 
future investigations. 

Overall, this assessment provided insight and guidance to improve NASA’s posture as a smart 
buyer for future LIDAR RelNav sensor procurements. Furthermore, it improved NASA’s 
familiarization with an understanding of each LIDAR sensors’ hardware and software 
(algorithm) functionality. By providing insight on design and performance of this existing 
family of LIDARs and their raw data processing RelNav algorithms, NASA gained an improved 
definition of LIDAR sensor performance/functional requirements for future sensor procurement 
specifications. 

LIDAR sensors are baselined for several of NASA’s, and its industry partners, future missions. 
LIDAR sensors perform the critical rendezvous RelNav sensing from initial long-range (several 
km) acquisition and tracking of the target spacecraft/body through terminal range/bearing 
measurements for proximity operations and for final alignment for docking/berthing/capture. 
LIDAR RelNav sensors will be used on the Orion Multi-Purpose Crew Vehicle (MPCV), 
expected to be used on the new generation of commercial crew and cargo vehicles being 
developed by industry. A LIDAR sensor was baselined for the guidance, navigation, and control 
(GN&C) subsystem on NASA’s Origins, Spectral Interpretation, Resource Identification, and 
Security Regolith Explorer Asteroid Sample Return Mission under development. Similarly, 
LIDAR sensors will be employed for RelNav sensing for future spacecraft servicing missions. 
Consequently, given this reliance on LIDAR sensors for future missions and the need to mitigate 
the technical risk of their use, NASA engineers and mission designers must better understand the 
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intrinsic functionality and performance characteristics of flash and scanning LIDAR sensors. 

This assessment of the LIDAR sensor data sets (obtained from multiple SSP DTO flight tests) 
will increase NASA’s understanding and knowledge of LIDAR sensor technology. The result of 
this assessment improves NASA’s familiarization with an understanding of the current 
generation of LIDAR sensor hardware and associated software-based data processing RelNav 
algorithms. 

Six SSP missions have supported DTOs in which on-orbit flight testing of three different LIDAR 
rendezvous sensors have been performed. From these six, four were assessed (two DTOs from 
STS-128 and STS-13 1 were considered to be of low value since they utilized older versions of 
the TriDAR sensor and therefore are not included in this assessment). Each DTO data set was 
analyzed by the sensor supplier or the organization that supported the DTO. However, a 
comprehensive, quantitative analysis of the individual performance of each sensor versus their 
individual DTO performance specifications had not been performed for all DTOs. The 
specifications and objectives for each DTO were defined by their respective PI. Table 5.0-1 
provides the mission summary for each DTO, sensor system, and key personnel. 


Table 5.0-1. LIDAR Sensor and DTO Summary 


Rel Nav LIDAR Sensor 

STS 

DTO 

Mission App 

DTO 

Principal 

Investigator 

NASA DTO 
Sponsor 

DragonEye (ASC) 

127 

(2009) 

SpaceX Dragon S/C 

Paul 

Wooster 

(SpaceX) 

Mike Horkachuck and 
Warren Reummele 
(COTS) 

DragonEye (ASC) 

133 

(2011) 

SpaceX Dragon S/C 

Paul 

Wooster 

(SpaceX) 

Mike Horkachuck and 
Warren Reummele 
(COTS) 

STORRM VNS 

134 

Orion MPCV S/C 

Heather Hinkel 

Howard Hu 

(Ball) 

(2010) 


(JSC/EG) 

(Orion) 

TriDAR 

135 

Orbital CygnusS/C 

Stephane Ruel 

Larry Schmidt 

(Neptec) 

(2011) 


(Neptec) 

(JSC/MV) 


NESC Request No.: TI- 11-00753 









NASA Engineering and Safety Center 
Technical Assessment Report 

Document #: 

NESC-RP- 

11-00753 

Version: 

1.0 

Title: 

Relative Navigation Rendezvous Sensor DTO 
Performance Evaluation 

Page #: 

15 of 103 


6.0 Background 

This section provides a high-level overview of the team’s approach and activities described in 
subsequent sections. At the beginning of the assessment, the team reached a consensus on 
analyzing the range and two components of bearing as compared to the BET for the performance 
metrics for all of the LIDAR sensors. The BET is not measured without error; therefore 
uncertainty in the BET was recognized and considered in the comparison analyses contained in 
this report. The DTO data sets were obtained from a draft release of reference 9; SpaceX- 
supplied data from the DragonEye DTO, and TriDAR data through a NASA/CSA formal 
agreement. The BETs (discussed in detail in a subsequent section) were generated for each 
DTO, only STS- 134 BET existed, and was used as the benchmark for comparison. A pre- 
processing step was perfonned to identify the targets and align the LIDAR sensor with the BET. 
A statistical comparison methodology was developed to compare LIDAR sensor-derived 
estimates of range and bearing with the BET. The analyses of the three sensors were conducted 
in a modular analysis utilizing multiple subteams to perform independent confirmatory analyses 
for each DTO. Figure 6.0-1 is a view into the orbiter payload bay that illustrates the location of 
the LIDAR sensors relative to the TCS. The BET is computed from TCS measurements through 
the RPOP Kalman filter, and the BET is used as the reference for comparison. Figure 6.0-2 is a 
view looking at the ISS docking ring and highlights some of the typical locations of reflective 
targets discussed throughout the report. 



Figure 6.0-1. View into Orbiter Bay Illustrating LIDAR Sensor Location 
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Figure 6.0-2. View into the ISS Docking Ring Illustrating Reflective Target Locations 

Figure 6.0-3 shows the ranges of each sensor and the sensors’ range bins that were analyzed 
during the assessment. The range bins were defined by the sensor developers to optimize sensor 
parameter settings by range. 


TriDAR 

DragonEye 

STORRM 



Range (meters) 

Figure 6.0-3. Overlay of NESC Assessment Ranges Analyzed (5 km extension of STORRM with 

yellow denoting as analyzed by the NESC) 


In the following sections, details of the analysis process are described followed by the results for 
the three LIDAR sensors. 
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6.1 BET Generation 

The BET is derived post-flight from multiple sources and is used as a benchmark for comparison 
to LIDAR sensor-derived range and bearing. An overview of the BET is provided in this 
section, see Appendix D, E, F, and G for additional details. The BET is used as the reference for 
comparison, and the development process is outlined in Figure 6.1-1, O denotes the state 
transition matrix, P denotes the state covariance matrix, and x denotes the [6x1] state vector. 
The equations of the smoother are given below [ref. 1]. 



A 


Reflector ID 
|! V| threshold 
TCS variances 


Post-Flight 

Analysis 


i*(+) 



B.E.T 


5x 




Figure 6.1-1. BET Development Process 

Let the final step be denoted by the subscript N. The smoothing algorithm begins at the last state 
and moves backward in time. The smoothed state vector at step k is estimated from: 

= xd+) + A t[x t+ i| W -x t+ i(-)] 
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where A k = P / .(+)0[Pj l (-) and x V A , = x v (+) for k = N - I 
The smoothed covariance matrix is computed from: 

V= p *( + ) +A *[ p ^- p ^(-)K 

where P V1;V = P v (+) for k = N - 1 

Flights to the ISS since STS- 1 14 included an R-bar pitch maneuver (RPM). During this 
maneuver, the orb iter rotated 360 degrees in pitch at 0.75 degree/second (deg/sec). There are no 
TCS measurements available to the RPOP Kalman filter throughout the rotational maneuver. 
The output of RPOP is input to the optimal smoother. Without measurements during this time 
interval, the optimal smoother is unable to maintain an accurate trajectory. Therefore, after TCS 
reacquisition following the RPM there is a discontinuity in the trajectory. 

For each flight that a BET is constructed, there is analysis required to determine the reflectors 
tracked by the TCS during the approach. RPOP has automatic reflector identification as a built- 
in feature, but does not reliably work at long range. 

After identifying the reflectors TCS tracked, the appropriate RPOP output file is adjusted and 
RPOP is run in “replay” mode. In replay mode, the data were input to RPOP during flight are 
processed by RPOP after the flight. In addition, during replay mode, the state transition matrix 
and the state covariance matrix are output, whereas in flight they were not output to save data 
storage space on the hard drive. 

RPOP assumes the ISS is fixed in LVLH attitude during the approach. To obtain a more 
accurate BET, the estimated ISS attitude from the ISS on-board software is used. The data sets 
were retrieved from the orb iter data reduction complex (ODRC). 

In addition to setting the playback data with appropriate reflectors that TCS tracked, the team 
varied some of the RPOP navigation initialization loads (I-loads) to obtain optimal performance. 
The RPOP I-loads were developed to allow the filter to work for a variety of realized 
performance from the TCS. Therefore, the variances of the measurements were chosen to be 
larger than the anticipated performance. When chosen in this way, if the TCS had modestly 
degraded performance on a particular flight, the filter would have been able to operate 
nominally. 

In developing the BET, estimates of the noise were generated because the TCS flight data were 
analyzed after most flights [ref. 2]. The team does not need to run the Kalman filter with the 
conservative I-loads developed for flight, but may tune the I-loads to account for the flight 
performance for which a BET is developed. 


NESC Request No.: TI- 11-00753 



NASA Engineering and Safety Center 
Technical Assessment Report 

Document #: 

NESC-RP- 

11-00753 

Version: 

1.0 

Title: 

Relative Navigation Rendezvous Sensor DTO 
Performance Evaluation 

Page #: 

19 of 103 


The quantization of the orbiter’s inertial measurement unit (IMU)-sensed velocity was 0.01049 
m/sec in each axis [ref. 3]. To guarantee an IMU bias, within its specification, is not used in 
RPOP state propagation, the IMU-sensed delta-velocity (AV) is used only when its magnitude 
exceeds 0.01829 m/sec. The Space Shuttle On-Orbit/Rendezvous Navigation Principal Function 
software used the same threshold [ref. 4]. 

Previous experience has shown that lowering the AV threshold to 0.01219 m/sec improves 
performance in constructing a BET, which was done for each of the flights analyzed during this 
study. 

In addition to the changes in TCS-related parameters and the AV threshold, the team used 
Kalman filter underweighting when the square root of the sum of the first three diagonal 
elements of the covariance matrix exceeds 10. During this time, the HPH r term in the 
calculation of the Kalman gain was increased by 20 percent [ref.4]. 1 

Finally, in executing RPOP in playback mode, the orbiter body-to-TCS coordinate 
transformation matrix was redefined. The TCS is mounted to the orbiter docking system (ODS). 
ODS pressurization results in deflecting the TCS about 0.9 degrees toward the forward orbiter 
bulkhead. The effect of this was the orbiter body-to-TCS frames were related by a rotation of 
only 89. 1 degrees, rather than the nominal value of 90 degrees. This effect is accounted for in 
constructing each BET for this assessment, but the nominal value (90 degrees) was used in the 
flight RPOP versions. 

6.2 ISS Retro-reflector Location 

The ISS has several retro-reflector assemblies located on various modules to aid incoming 
vehicles in their relative navigation determination through the use of LIDAR units. In support of 
the orbiter TCS, the reflectors were cataloged for quicker processing and identification within 
RPOP. Figure 6.2-1 shows the general location of the reflector assemblies and their 
corresponding field of regard (FOR). The color-coded traces highlight the FOR of each 
reflector. Table 6.2-1 lists the same reflectors and matches their TCS identification number to 
their location and pointing direction. This infonnation is utilized throughout this report when 
discussing various reflectors investigated. ISS modules and appendages cause blockage to some 
of the reflectors, thus reducing their effective FOR. The blockage is seen in the Figure 6.2-1 by 
the segmented and/or shortened arcs. 


1 Here, H denotes the partial derivative of the measurement with respect to the relative state vector and P denotes the 
state covariance matrix. 
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Figure 6.2-1. ISS Retro-reflector Locations and FOR 


Table 6.2-1. ISS Retro-reflector Information 


TCS # 

Component Name 

Pointing 
Direction * 

FOR** 

(degree) 

Reflector Type 

1 

PMA2 Node2 FWD, Planar 

+X 

30 

Single Planar 

2 

PMA2 Node2 FWD, Hemi 

+z 

90 

Hemispherical 

3 

FGB FWD, Hemi 

+x 

20 

Hemispherical 

4 

FGB AFT, Hemi 

-X 

40 

Hemispherical 

5 

PMA3 / PMM Node 1 Nadir 

+z 

30 

Single Planar 

6 

SM AFT 

-X 

30 

Multiple Planars 

7 

JEM A {JPM (2) in figure} 

+z 

90 

Hemispherical 

8 

JEM B {JPM (2) in figure} 

+z 

90 

Hemispherical 


* Pointing direction with respect to the ISS LVLH reference frame (as per Figure 6.2-1) 
** FOR, half-angle in degrees 
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6.3 LIDAR Sensor Attitude Alignment 

The vehicle state inputs from the BET for each DTO flight could account for misalignments 
between the predicted and actual reflector centroids. The ISS attitude was held fixed in the 
LVLH reference frame. The second factor was the alignment of the detector behind the sensor 
optics. For the STORRM VNS and TriDAR, the translational alignments were mapped to the 
detectors but the rotational alignments were unknown. For the DragonEye, the detector 
translational and rotational alignments to the sensor optics were unknown. A sensor-to-BET 
attitude alignment analysis may provide some of the missing information. The last factor 
emphasizes the importance of precise knowledge of the sensor location with respect to the orbiter 
structure. For the DragonEye flights, the sensor location was not surveyed and may have 
contributed errors to the predicted reflector centroids. 

To correct for some of the identified misalignments and to employ the predicted reflector 
centroid data for easier BET comparison, two auxiliary features were included with the reflector 
visibility tool: the sensor to BET attitude misalignment correction and centroid matching for 
reflector identification and sorting. 

The sensor to BET attitude misalignment was computed to achieve maximum agreement 
between the predicted data and actual sensor measurements. A single angular misalignment 
matrix was computed for the entirety of the data collected on each DTO mission (STS- 127, 
STS-133, and STS-134) using the Davenport q-method solution to Wahba’s problem. A 
shortcoming of this method is when the translational misalignment is unknown, which is the case 
for the DragonEye data. Additionally, performing this alignment correction on smaller sections 
of the trajectory data produce better agreement between the predicted and actual centroids. 

6.4 Reflector Visibility Identification 

The reflector visibility tool was used to generate the predicted reflector locations with respect to 
the sensor frame for the STORRM VNS on STS-134 and DragonEye on STS-127 and STS-133. 
A detailed description of this tool developed for STORRM is provided in Appendix I. Through a 
series of coordinate frame transformations, the tool used the ISS and orbiter relative position and 
attitude as obtained from RPOP, reflector locations with respect to the ISS structure frame, and 
sensor location with respect to the orbiter body frame (OBF) to check for line of sight visibility 
between reflectors and the sensor. The locations of STORRM reflective elements were obtained 
through a photogrammetric survey performed on STS-134. The tool accounted for reflector 
obscuration from ISS structure. However, the tool did not account for reflector return signal 
strength and signal interference. 

Reflector visibility was contingent on the sensor being in the reflector’s unobscured FOR which 
is target specific and the reflector being in the sensor’s field-of-view (FOV). For the STORRM 
VNS, a third requirement for visibility was the reflector must be illuminated by the VNS laser 
since the laser field of illumination was commanded between 12- and 20-degrees. Once these 
checks were satisfied, the reflector was projected on the sensor image plane. Figure 6.4-1 shows 
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a screenshot of reflector visibility tool graphical user interface (GUI). The output panel includes 
the predicted reflectors in the intensity image, the relative motion plot of the orbiter with respect 
to the ISS center of gravity (CG) in a LVLH reference frame, time tags, calculated and raw TCS 
and state vector derived range, vehicle position and attitude inputs, and a table of visible 
reflectors and their predicted pixel locations. 



Figure 6.4-1. Reflector Visibility Tool GUI 

The tool’s accuracy in predicting exact reflector pixel location was dependent on the following 
factors: 

1 . Vehicle state inputs and misalignments. 

2. Detector alignment to sensor optics. 

3. Knowledge of sensor location with respect to the OBF. 

After correcting for attitude misalignment, the predicted centroid data was used to match sensor 
centroids with ISS reflectors. The closest flight measurement to the predicted centroid was 
identified as that particular reflector. Through this process, the sets of centroids were sorted per 
reflector, which streamlined the process of comparing sensor to BET measurements. 
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Coordinate Frame Definitions 

Reference 21 details the orbiter vehicle and body frames, the STORRM VNS camera frame, and 
the VNS image plane. The ISS document in reference 22 describes the ISS LVLH frame and the 
Space Station Analysis Coordinate System (SSACS). This section discusses the pertinent 
coordinate frames used in the tool that are not described in the aforementioned reference 
documents. These coordinate frames are the VNS transmit optics frame and the local reflector 
frame. 

STORRM VNS Transmit Optics Frame 

The STORRM VNS transmit optics frame is parallel to the VNS camera frame. The frame 
origin is a variable input in the run configuration and its values are with respect to the VNS 
camera frame. The intent of this frame was to improve simulation fidelity of the VNS receive 
and transmit optics. The specular nature of the corner cube reflector return causes issues with 
the bi-static configuration of the VNS receive and transmit optics. However, this phenomenon 
was not fully modeled and implemented in the tool since it is sufficient to rely on line of sight for 
STORRM. 

Local Reflector Frame 

The local reflector frame is fixed to the reflector’s front plane with the positive x-axis along the 
boresight and the y- and z-axes lying on the reflector’s front plane to complete a right-handed 
Cartesian coordinate frame (Figure 6.4-2). 
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STORRM VNS and Reflector Line of Sight Vectors 

Given the known values from the run configuration and input data, the goal is to obtain the line 
of sight vectors between the reflectors and the STORRM VNS to determine visibility. 

Figure 6.4-3 shows the process of computing the line of sight vectors from the kn own vectors. 


Retro location with 
respect to SSACS 


Retro location with 
respect to ISS CG/LVLH 

Retro " 

Retro/VNS line 
of sight vector 

VNS location with 
respect to ISS CG/LVLH 

VNS location with respect 
to Orbiter Structure Frame 

Orbiter 

Structure 

Frame 



ISS CG with 
respect to SSACS 


— Known vectors 

— - Computed vectors 

— Line of sight vector 


Orbiter rei pos 
vector 

VNS location with 
respect to Orbiter CG 


Orbiter CG with respect 
to Orbiter Structure Frame 


Orbiter CG 


Figure 6.4-3. Frame Transformation Sequence 


Transformation from one frame to another was obtained using: 


X B = MgX^ 


— > 


X- 



x- 

y 


T B C/B 


y 

z 


-®lx3 1 


z 

-1- 

B 


-1- 


where X A is the location of the object in frame A and X B is the location of the object in frame B. 
T B is the transformation matrix from frame A to frame B, and t A / B is the location of the origin of 
frame A with respect to frame B [1], 


Reflector Obscuration Model 


The reflector obscuration model requires the line of sight vector from the reflector to the 
STORRM VNS. Given how the obscuration data is structured, the line of sight vector is rounded 
to the nearest 10-degree increment in the reflector y-z plane. The corresponding minimum and 
maximum FOR angles are then used to determine visibility or obscuration. Figure 6.4-4 shows 
the coordinate frame convention for the reflector obscuration data. The y-z plane of the reflector 
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is divided into 10-degree increments with the 0-degree lying on the y-axis and the 90-degree 
mark lying on the z-axis. The reflector FOR angle is measured with zero degrees at the 
reflector’s boresight (x retr o) and increasing towards the y-z plane (90 degrees on the y-z plane). 



Figure 6.4-4. Reflector Obscuration Data Coordinate Frame Convention 
Transformation to STORRM VNS Image Frame 

Once the STORRM VNS to reflector line of sight is obtained and the obscuration checked, the 
reflector location can then be transformed into the VNS image frame using: 



it 



0 

Up 

-X- 

u /p — K/p x /p 

V 

= 

0 

s y 

Vp 

y 


-1- 

ip 

- 0 

0 

1 

LfJ 


where s x and s y are the x-axis and y-axis scale factors in units of pixel/length, and u p and v p is 
the location of the VNS boresight with respect to the VNS image [u, v] coordinate system [1]. 

6.5 Statistical Analysis Methodology 

Sensor performance is based on a comparison to the BET. In this assessment, the BET is given 
as a CG-to-CG state vector of relative position and velocity. From the CG-to-CG position 
vector, the sensor-to-reflector position vector through coordinate transformation may be 
computed. The BET-estimated range, horizontal, and vertical measurements are functions of the 
BET-based sensor-to-reflector vector. Figure 6.5-1 shows the computation of the sensor-to- 
reflector position vector. 
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Sensor 



Figure 6.5-1. Measurement Geometry 


The BET is expressed in LVLH coordinates based on Figure 6.5-1 : 


( LVLH) (L VLH) .ryLVLH (OrbiteiBody) rrLVLH ( ISSbody ) 

sensorto reflector ISSCGtoOrbiterCG ^Orbiterbodjr OrbiterCGto sensor ISSbody ISSCG to reflectoi 

The vector ^tu'oiioreflecto, i s transformed to sensor coordinates. The BET-estimated measurements 

( senson 

are computed from the definition of the measurement in terms of r senS orto reflector These definitions 
vary from sensor to sensor. 

The estimated measurement errors are the sensor-derived measurement minus the BET-derived 
measurement. 

Before computing estimated errors, the measurements must be sorted so each measurement is of 
the same reflector. Because the BET and sensor had different data rates, interpolation is 
necessary to synchronize the measurements. As an example, for STORRM the BET is given in 
approximately 4-second intervals, and the sensor data is given in 0.2-second intervals. Linear 
interpolation is used in this analysis. It has been shown that the error introduced by interpolation 
is small (see Appendices A, B, and C). 

During each rendezvous of the orbiter with the ISS from STS-1 14 through STS-135, the orbiter 
executed the R-bar pitch maneuver at about 183 meter on the positive R-bar. During most of this 
time, the orbiter’s TCS and DTO sensors were not pointed at the ISS and thus no measurements 
were produced. At the beginning and end of the RPM, the TCS was able to track a reflector on 
the ISS, but the accuracy of the BET is compromised during these intervals. Therefore, 
measurements by the DTO sensor at the beginning and end of the RPM are not included in this 
analysis because the estimated errors during these times are not reliable. 

Any bias in the BET potentially adds to the estimated bias. Unless corrected for the bias, the 
estimated bias may be an underestimate. As explained in references 5 and 6, the TCS mounting 
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and misalignment biases are unknown, but estimated to be no worse than 1 degree. Because of 
this, although the team reports deviations between the BET-estimated horizontal and vertical 
measurements and the DTO sensor measurements, the team did not attempt to separate these 
deviations into bias components attributable to the DTO sensor and the BET. 

For range measurements, the team estimated biases of the sensor measurements. The team 
computed the estimated bias by a two-step procedure. First, the team fit a polynomial ordinary 
least squares (OLS) regression to the estimated range errors. Next, the team computed the 
autocorrelation at lag 1 in the residuals from the OLS regression and then relit using generalized 
least squares (GLS) regression, accounting for the autocorrelation 2 [refs. 7, 8]. 

The DTOs tested scanning and flash LIDARs. The scanning LIDARs employ at least one set of 
gimbaled mirrors coupled with a narrow light source (typically an infrared (IR) laser operating in 
a pulse or continuous waveform manner). The two mirrors operate concurrently to produce a 
predefined scan pattern, providing mirror azimuth and elevation measurements. The scan pattern 
can be variable and adjusted based on moding and light energy returns. The TriDAR is a 
scanning LIDAR. Flash LIDARs employ a multi-pixel detector array and a light source 
(typically an IR laser). The laser emits a short pulse, which transmits through space and reflects 
off the target vehicle and returns. The laser energy returns and strikes the detector through a set 
of optics that steer/focus the light onto the detector. Each pixel measures the time of flight and 
laser energy intensity. The DragonEye and STORRM VNS are two variants of flash LIDARs. 

7.0 Data Analysis 

7.1 STORRM 

7.1.1 Overview of STORRM DTO 

The STORRM DTO flew aboard the Endeavour on STS- 134 in May- June 2011. STORRM was 
designed to characterize the performance of the VNS flash LIDAR and docking camera (DC) 
being developed for the Orion MPCV Program. In addition, STORRM was designed to collect 
data from these sensors to mitigate the loss-of-mission risk in Orion test flights and to increase 
the sensor technology readiness level (TRL). Because the present report is concerned only with 
LIDARs, only the VNS will be reviewed. Information on the DC may be found in other 
STORRM-specific documentation [ref. 9]. 

The STORRM VNS will be the primary navigation instrument used by the Orion vehicle during 
RPOD starting at 5 km from the target vehicle. To assess sensor performance during the DTO, 
the VNS and DC were placed in the STORRM sensor enclosure assembly (SEA). The SEA was 


2 Autocorrelation is a measure of randomness ascertained by computing the correlation, or systematic relationship, 
between data values at varying time lags. If random, such autocorrelations should be near zero for any and all time- 
lag separations. 
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then mounted in the orbiter payload bay directly next to the TCS, a scanning LIDAR system 
used during proximity operations. A BET utilizing TCS measurements served as the benchmark 
for VNS post-flight analysis. The VNS data collection opportunities occurred on flight day 3 
(FD3) during rendezvous and docking to the ISS and flight day 15 (FD15) during undock, fly- 
around, re-rendezvous and final separation. The re-rendezvous, designed to mimic the planned 
Orion rendezvous profile, differed from the nominal Shuttle trajectory. Throughout all of these 
mission phases, VNS data were produced and recorded at a rate of 30 Hz. These data were then 
combined during post-flight analysis to produce reflector range and bearing measurements at a 
rate of 1 or 5 Hz. More specifically, the range and bearings at each measurement time were 
generated through a NASA-developed centroiding algorithm [ref. 10] designed to deal directly 
with reflectors in flash LIDAR imagery. 

The prototype VNS tested during the STORRM DTO had eight different range bins. Using the 
phonetic alphabet, these are referred to as range bins Alpha through Hotel. Each range bin 
corresponded to a different set of sensor settings (e.g., gain, sensor internal timing, etc.) to 
improve its performance over a specific range. Therefore, the VNS results presented 
performance statistics and observations as a function of range bin. 

In general, STORRM VNS performance during rendezvous, docking, undocking, fly-around, and 
re-rendezvous was as expected. Minor issues were encountered during rendezvous, but timely 
response from the STORRM team put on-board contingency procedures in motion resulting in 
minimal disruption to STORRM activities. STORRM collected 380 GBytes of VNS data while 
on-orbit. The STORRM team was able to observe through periodic data snapshots that the VNS 
performed as expected in terms of sensor moding (detector gain settings) and laser firing. The 
VNS performed well in imaging the ISS and exceeded expectations for its long-range ISS 
acquisition. 

The overall STORRM VNS mission objectives were to test the on-orbit performance on three 
different trajectories: 

1 . A nominal orbiter RPOD trajectory to the ISS. 

2. A nominal orbiter undocking trajectory from the ISS. 

3. An Orion-like flight rendezvous/proximity operations trajectory to the ISS. 

To meet the third objective, the space shuttle separated from the ISS and then performed an 
unprecedented re-rendezvous with the ISS with an approach of approximately 291 meters 
(956 feet). 

Because the results discussed in this report deal only with the rendezvous and not the undock or 
re-rendezvous, focus in the subsequent sections will only be on the rendezvous performed on 
STS- 134 FD3. Re-rendevous analyses are included in reference 9. The team decided to exclude 
it from this assessment since the approach was representative of performance. A graphical 
depiction of the STORRM rendezvous is shown in Figure 7.1-1. A summary of the VNS-related 
mission objectives are provided in Table 7.1-1. 
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Rendezvous Relative Motion Plot 



Figure 7.1-1. Relative Motion Profile for Nominal Orbiter Rendezvous and Docking 
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Table 7.1-1. STORRM VNS-related Mission Objectives 


Flight Phase 

# 

VNS Objective 


VOIP 

Characterize the ISS in the VNS wavelength (hot spots, 
obscuration, reflectance map) 

Rendezvous / Prox Ops 
(6 km - 3 km) 

V02P 

Determine initial acquisition range 


V03P 

Prove operation with large relative velocities 


V04P 

Characterize the ISS in the VNS wavelength (hot spots, 
obscuration, reflectance map) 

Prox Ops (3 kin -50 m) 

V05P 

Collect data through known break track and reacquire 
conditions 

V06P 

Prove tracking through accelerations while maneuvering 


V07P 

Demonstrate transition between 3-degree-of-freedom 
(DoF) and pose modes 


V08P 

Allow for overlap with ground testing facilities 

Final Approach (50 m - 
dock) 

V09P 

Collect data to support pose calculation 


V10P 

Characterize the ISS pressurized mating adaptor 2 
(PMA2) augmented docking target 


vnp 

Perfonn calculations on raw data (R and I measurements) 
to determine geometric centroids to reflective elements 

All Flight Phases 

V12P 

Perform calculations on raw data to determine range and 
horizontal/vertical angles to target 


V14P 

Characterize noise and bias 


The range bin for STORRM are detailed in Table 7.1-2, the green highlighted boxes were 
assessed by the team. 
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Table^J^2^TORRMRan^eBin^ 


STORRM 

Bin 

Range Bounds 
(m) 

Hotel 

0-30 

Golf 

30-81 

Foxtrot 

81 - 162 

Echo 

162 - 324 

Delta 

324 - 648 

Charlie 

648 - 1300 

Bravo 

1300-2590 

Alpha 

2590 - 5760 


Summary of the STORRM VNS Performance from Reference 9 

To obtain information useful for assessing the STORRM VNS’s performance as a navigation 
sensor, a number of post-processing steps were perfonned on the raw data stored on the 
STORRM data recording units (DRUs). First, a range calibration algorithm supplied by Ball 
Aerospace Technologies Corporation was applied to the raw VNS images, which resulted in the 
“range corrected” images. Then, centroiding and reflector identification algorithms were used to 
identify candidate reflectors within the VNS imagery. For the bulk of the rendezvous, this 
algorithm combined six consecutive VNS images (collected at 30 Hz) to produce centroid 
measurements to candidate reflectors at a rate of 5 Hz. These candidate reflectors were 
compared to what the VNS was expected to see based on the BET. Using this comparison 
information, each candidate reflector could then be matched to a kn own reflector or be identified 
as a spurious reflection. Finally, after each candidate reflector in each VNS image was 
identified, a statistical analysis was perfonned to summarize the performance relative to the 
reference trajectory derived from the BET on the range and bearing of each reflector. A 
comparison with the BET, an enlargement of range bin Hotel (the closest range bin) is shown in 
Figures 7.1-2 and 7.1-3. 

The range performance was noted to be out of specification in range bin Echo (162-324 meters). 
This anomaly is described in reference 9 and was attributed to VNS settings for that particular 
range bin. However, the nature of the problem was not fully resolved. Therefore, the team did 
not perfonn any further analysis on this range bin. 
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Figure 7.1-2. Time History of Centroid [u, v] Coordinates from STS-134 Docking: Range Bin Hotel 



NESC Request No.: TI- 11-00753 



NASA Engineering and Safety Center 
Technical Assessment Report 

Document #: 

NESC-RP- 

11-00753 

Version: 

1.0 

Title: 

Relative Navigation Rendezvous Sensor DTO 
Performance Evaluation 

Page #: 

33 of 103 



15 10 

port to port tango, motors 


Controid 

— STORRM1 

— STORRM2 
STORRM3 
ST0RRM4 
STORRM5 

— PMA2PUnar 


STORRM reflectors 
outside of FOV 

I > 


Figure 7.1-3. Time History of Centroid Ranges from STS-134 docking: Range Bin Hotel 

Reference 9 captures an exhaustive analysis of the noise statistics for each range bin in all phases 
of flight. 

In the following subsections, the VNS range performance will be analyzed, in order of range bin 
(Charlie, Delta, and Hotel), and following that will be descriptions of the bearing performance 
analysis. 

7.1.2 STORRM VNS Range Measurement Performance Analysis 

Based on the statistical methodology described in Section 6.5, this section presents the 
performance analysis of the STORRM VNS by comparing it to the BET. The analysis is 
presented by range bin starting with the farthest distance from the ISS rendezvous. Range 
comparisons are presented first, followed by bearing comparisons. 
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Figure 7.1-4 shows the range measurements to TCS reflector 2 co-plotted with the corresponding 
sensor-to-reflector range computed from the BET for range bin Charlie. There are two extreme 
range errors seen in Figure 7.1-4. 


1800 

1600 


STS- 134 VNS Range Measurement* 
Compared to B.K.T .-Estimated Measurements 
Tracking TCS reflector 2 in range bin charlie 


• 

VNS 


- 13 E.T. 



Figure 7.1-4. STS-134 STORRM VNS Range Measurements of Reflector 2 with BET-estimated 

Range 


Figure 7.1-5 shows the estimated range errors in range bin Charlie with two outliers omitted. 

The errors are designated as “estimated” because the BET does not precisely match the true 
range. From Figure 7.1-5, the estimated range error does not behave as white noise. The green 
line shows the estimated bias in the STORRM VNS range measurements in range bin Charlie. 
The orange bands above and below the estimated bias are 90 percent confidence bands. 

These bands are the sum of the Working-Hotelling bands obtained by generalized least-squares 
regression and the confidence bands in the BET. Working-Hotelling confidence bands that 
provide a confidence limit for the entire regression line, in contrast to the confidence interval for 
a single predicted value, were considered by the team to provide a conservative estimate of 
uncertainty. At any range the orange bands exclude zero, the corresponding estimated bias is 
statistically significant at significance level 0.10. The reason why the data lies outside the 
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confidence bands is because the bands apply to the estimated bias, not to the overall scatter in the 
estimated errors. 

Figure 7.1-5 shows the estimated range bias in range bin Charlie was relatively small, being at 
most about 2.5 meter at a range of nearly 1300 meters. 


STS-134 Mean VNS Range Krror n. Range 
90% confidence bands: Tracking TC'S reflector 2 in range bin charlie 
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Figure 7.1-5. STS-134 STORRM VNS Bias with 95 percent Working-Hotelling and 90 percent 

Composite Bands 


The estimated standard deviation of the STORRM VNS range measurement noise is given in 
Table 7.1-3 by multiple methods. The most accurate estimate of the noise is the GLS regression 
of second degree with two outliers omitted. In that case, the standard deviation is 0.643 meters. 
A 90 percent confidence interval on the standard deviation (again omitting the two outliers) is 
(0.626, 0.661) meter. This interval is short because the sample size is large (n = 2063). The data 
are not normally distributed. Therefore, the interval is computed by a bootstrap method 
[refs. 11, 12]. 
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Table 7.1-3. Estimated Standard Deviations ofSTORRM VNS Range Measurement Noise; 


Tracking TCS Reflector 2 in Bin Charlie; N = 2063 + Two Outliers 


Method 

Estimated standard 
deviation (m) 

Degree 2 OLS regression of diff. from BET; 
with 2 outliers 

19.859 

Second-difference method including the 2 
outliers 

19.497 

Degree 2 OLS regression of difference from 
BET 

0.643 

Degree 2 GLS regression of difference from 
BET 

0.643 

Quadratic OLS regression of range 
measurement versus time 

0.979 

Loess of range versus time with smoothing 
parameter 0.20 

0.602 

Second-difference method (assumes white 
noise) 

0.305 

Second-difference method (assumes auto- 
regressive (AR) (1) process) 

0.303 


The team found the variance of the error is nearly constant as a function of range. 

STS-134 STORRM VNS Range Measurement Performance in Range bin Delta 

Figure 7.1-6 shows the STORRM VNS range measurements to TCS reflector 2 in range bin 
Delta, co-plotted with the corresponding sensor-to-reflector range computed from the BET. 
Numerous extreme errors can be seen in Figure 7.1-6. 
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Tracking TCS reflector 2 in range bin delta 



Mission-elapsed time (sec) 

Figure 7.1-6. STS- 134 STORRM VNS Range Measurements to Reflector 2 with BET-estimated 

Range 

Figure 7.1-7 shows the estimated STORRM VNS range errors in range bin Delta with the 
48 largest extremes omitted, and the estimated bias with the extremes omitted and the 
corresponding 90 percent confidence bands. The bias is statistically significantly different from 
zero (at level 0.10) at all ranges. Details of the statistical analysis are provided in Appendix A. 
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Figure 7.1-7. STS-134 STORRM VNS bias with 95 Percent Working-Hotelling and 90 Percent 

Composite Bands 


Only one extreme estimated error was negative (i.e., -62.56 meters). All the other extreme 
errors were positive. The remaining 47 extremes were not normally distributed. It was 
hypothesized the extremes were distributed as a lognormal. Applying the Anderson-Darling 
goodness-of-fit test [ref. 13], the hypothesis is accepted at significance level 0.05. 

The noise, calculated using the standard deviation of the residuals was characterized. The 
estimates calculated using multiple methods are shown in Table 7.1-4. Note the large difference 
in standard deviation with and without inclusion of the 48 extreme values. 
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Table 7.1-4. Estimated Standard Deviations ofSTORRM VNS Range Measurement Noise; 
Tracking TCS Reflector 2 in Range Bin DELTA; N= 1956 


Method 

Estimated standard deviation (m) 

Direct calculation of standard deviation from 1956 range 
errors 

4.988 

Degree 3 OLS regression fit omits 48 outliers, but 
residuals include outliers 

4.970 

Second-difference method without omitting extremes 
(n = 1913) 

4.706 

Degree 3 OLS regression of difference from BET; omit 48 
outliers 

0.479 

Degree 3 GLS regression of difference from BET; omit 48 
outliers 

0.473 

Second-difference method omitting 162 extremes 
(n = 1751) 

0.145 


A two-sided 90 percent confidence interval on the standard deviation of the noise including the 
outliers was 4. 1 18 and 6.066 meters. A 90 percent confidence interval on the standard deviation 
of the noise with the outliers excluded is (0.442, 0.53 1) meter. The variance changes modestly 
as a function of range, but not enough to be of practical significance. 

STS-134 STORRM VNS Range Measurement Performance Tracking TCS Reflector 1 in 
Range Bin Hotel 

Figure 7.1-8 shows the STORRM VNS range measurements to TCS reflector 1 in range bin 
Hotel, co-p lotted with the corresponding sensor- to-reflector range computed from the BET. 
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Figure 7.1-8. STS-134 STORRM VNS Range Measurements to Reflector 1 with BET-estimated 

Range 


Figure 7.1-9 shows the estimated range measurement errors when STORRM VNS tracked 
reflector 1 with the estimated bias and its 90 percent confidence bounds. The range bias is 
statistically significantly different from zero at all ranges (at level 0.10). Reference 9 notes a 
1 .918-meter bias was observed in ground testing in range bin Hotel. The estimated bias seen in 
Figure 7.1-9 is close to this number though the bias varied with range. 
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STS-1J4 Mean VNS Range Error w. Range 
90% confidence bands: Tracking TC'S reflector 1 in range bin hotel 
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Figure 7.1-9. STS-134 STORRM VNS Bias with 95 percent Working-Hotelling and 90 percent 

Composite Bands 

The residuals may be well-modeled by a normal distribution. Application of an Anderson- 
Darling goodness-of-fit test [ref. 13] results in accepting the hypothesis of normality at 
significance level 0.05. 

The standard deviation of the residuals was characterized and the estimates are shown in 
Table 7.1-5. Note the estimates obtained from the second-difference method were lower than 
those obtained by the regression methods. This is consistent with TCS experience [ref. 2]. In 
either case, the estimated standard deviation of range noise was small. 
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Table 7.1-5. Estimated Standard Deviation of the STORRM VNS Range Measurement Noise; 


Tracking TCS Reflector 1 in Bin Hotel; N = 1138 


Method 

Estimated standard deviation (m) 

Degree 1 OLS regression of difference from 
BET 

0.031 

Degree 1 GLS regression of difference from 
BET 

0.031 

Second-difference method (n = 1 130); assume 
white 

0.012 

Second-difference method (n = 1 128); assume 
AR(1) 

0.014 


A two-sided confidence interval on the standard deviation of the noise while the STORRM VNS 
tracked reflector 1 is (0.030, 0.032). This is based in the residuals from a GLS regression. The 
standard deviation of the noise is nearly constant as a function of range in range bin Hotel. 

STS-134 STORRM VNS Bearing Measurement Performance in Range bin Charlie 

There is a bias in the vertical measurements relative to the BET. As mentioned in references 
5 and 6, the TCS bearing and misalignment biases are unknown. They may be as large as 
1 degree, though that is probably a worst case. A rotation matrix was computed to attempt to 
align the BET and the STORRM VNS measurements [ref. 14]. It is not possible to separate the 
bias in the BET from the bias in the VNS bearing. The magnitude of the deviation between VNS 
and the BET was reported, but it was not ascribed to the VNS bias. 

Noise estimates are made with and without having applied the alignment, though plots of the 
estimated errors are shown only for the case that the transformation is used. Transformation to 
the bearing measurements was applied to reflector 2 in range bin Charlie. The result for the 
horizontal measurements are shown in Figure 7.1-10 and the corresponding estimated errors are 
shown in Figure 7.1-11. 
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Figure 7.1-10. STORRM VNS Horizontal Measurements Compared to Altered BET Estimates 
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Figure 7.1-11. Estimated STORRM VNS Horizontal Measurement Error; Tracking 

TCS Reflector 2 
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A co-plot of STORRM VNS vertical measurements and the BET-estimated measurements are 
shown in Figure 7. 1-12. The estimated errors are shown in Figure 7.1-13. 


STS- 134 VNS Vertical Measurements 
Compared to BJC.T .-Estimated Measurements 
Tracking Tt'S reflector 2: includes B.K.T.-to-sensor adjustment 
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Figure 7.1-12. STORRM VNS Vertical Measurements Compared to Altered BET Estimates 
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STS- 134 VNS Vertical Measurement Krror 
Tracking TC'S reflector 2: bin Charlie: includes B.K.T .-to- sensor adjustment 
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Figure 7.1-13. Estimated STORRM VNS Figure 7.1-9 Vertical Measurement Error; Tracking TCS 

Reflector 2 

Table 7.1-6 gives estimates of the mean differences and standard deviations of the bearing errors 
with and without the alignment of the BET to the data. 


Table 7.1-6. Mean Differences and Standard Deviations of Bearing Errors; STORRM VNS 
Tracking TCS Reflector 2 in Range Bin Charlie (N = 2065) 



Mean diff. (deg) 

Std. Dev. 

Horizontal, not aligned 

0.001 

0.015 

Horizontal, aligned 

-0.001 

0.014 

Vertical, not aligned 

0.277 

0.032 

Vertical, aligned 

0.023 

0.032 


Ninety-percent confidence intervals on the standard deviations computed from the aligned values 
are given (0.0136, 0.0142) degree for the horizontal measurements, and (0.0316, 0.0325) degree 
for the vertical measurements. 
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The autocorrelation coefficients at lag 1 for the horizontal and vertical measurement residuals are 

•2 

0.9739 and 0.9880, respectively. These coefficients were computed without accounting for 
missing data. 

STS-134 STORRM VNS Bearing Measurement Performance while Tracking TCS 
Reflector 2 in Range bin Delta 

Figure 7.1-14 shows the horizontal angle measurements from the STORRM VNS to TCS 
reflector 2 in range bin Delta, co-plotted with the corresponding estimates computed from the 
BET. Figure 7.1-15 shows the estimated errors in the horizontal angle measurements. A co-plot 
of the vertical measurements and the vertical measurements computed from the BET is shown in 
Figure 7.1-16. The estimated vertical angle errors are shown in Figure 7.1-17. 
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Figure 7.1-14. STORRM VNS Horizontal Measurements Compared to Altered BET Estimates 


3 The computation of a sample autocorrelation coefficient assumes the data are equally spaced in time with no 
missing data. The VNS data include numerous instances for which this is violated. No attempt was made to account 
for the missing data in calculation of the sample autocorrelation coefficient. 
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Figure 7.1-15. Estimated STORRM VNS Horizontal Measurement Error; Tracking 

TCS Re dec tor 2 
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STS- 134 VNS Vertical Measurements 
Compared to BJC.T .-Estimated Measurements 
Tracking TCS reflector 2: includes B.K.T.-to-*cnsor adjustment 
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Figure 7.1-16. STORRM VNS Vertical Measurements Compared to Altered BET Estimates 
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The interval of range over which the fourth midcourse maneuver (MC4) executed by the orbiter 
occurs is indicated in Figure 7. 1-17. Translational acceleration has been kn own to result in 
increases in errors in bearing. In this case, larger changes in vertical measurement error occur 
subsequent to the MC4 maneuver. 


STS-l.M VNS Vertical Measurement Error 
Tracking TCS reflector 2: bin delta: includes B.K.T.- to- sensor adjustment 
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Figure 7.1-17. Estimated STORRM VNS Vertical Measurement Error; Tracking TCS Reflector 2 


Table 7.1-7 gives estimates of the mean differences and standard deviations of the bearing errors 
with and without the alignment of the BET to the data. 

Table 7.1-7. Mean Differences and Standard Deviations of Bearing Errors; STORRM VNS 
Tracking TCS Reflector 2 in Range Bin Delta; N = 1956 



Mean diff. (deg) 

Std. Dev. 

Horizontal, not aligned 

0.003 

0.030 

Horizontal, aligned 

-0.014 

0.028 

Vertical, not aligned 

0.293 

0.086 

Vertical, aligned 

0.039 

0.087 


Ninety-percent bootstrap confidence intervals (see reference 12 for more details on the method) 
on the standard deviations of the bearing measurement noise, computed from the aligned values, 
are (0.027, 0.029) degree and (0.082, 0.092) degree for the horizontal and vertical measurements, 
respectively. 
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The sample autocorrelation coefficients at lag 1 for the estimated horizontal and vertical 
measurement errors are 0.9852 and 0.9945, respectively. 

The standard deviation of the vertical noise was greater than the horizontal noise. There appears 
to be a time-varying bias with the noise relatively small based on examining Figure 7. 1-17. The 
standard deviation of the residuals is 0.01 1 based on fitting the vertical measurement error at 
ranges less than 533 meter with a fourth-degree OLS regression. 

Alternatively, the team may treat the vertical errors when the range is less than 533 meter as an 
AR(1) process. From this point of view, the errors may be modeled with an exponentially 
correlated random variable of standard deviation 0.066 degree and time constant 62 second. 

STS-134 STORRM VNS Bearing Measurement Performance while Tracking TCS 
Reflector 1 in Range bin Hotel 

Figure 7.1-18 shows the horizontal angle measurements from STORRM VNS to TCS reflector 1 
in range bin Hotel, co-plotted with the corresponding estimates computed from the BET. The 
estimated horizontal measurement errors are shown in Figure 7.1-19. 


STS-134 VNS Horizontal Measurements 
Compared to B JC.T.-Hstimated Measurements 
Tracking TCS reflector 1: includes B Jt.T.-to-sensor adjustment 
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Figure 7.1-18. STORRM VNS Horizontal Measurements Compared to Altered BET Estimates 
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STS- 134 VMS Horizontal Measurement Error 
Tracking TCS reflector I; bin hotel: includes B.E.T.-to-sensor adjustment 



Scncor-to- reflector range (m) 


Figure 7.1-19. Estimated STORRM VNS Horizontal Measurement Error; Tracking 

TCS Reflector 1 


Figure 7.1-20 shows the vertical angle measurements from STORRM VNS to TCS reflector 1 in 
range bin Hotel, co-plotted with the corresponding estimates computed from the BET. The 
estimated vertical measurement errors are shown in Figure 7.1-21. 
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STS- 134 VNS Vertical Measurements 
Compared to B .E.T .-Estimated Measurements 
Tracking TCS reflector 1: includes B.K.T.-to-sensor adjustment 
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Figure 7.1-20. STORRM VNS Vertical Measurements Compared to Altered BET Estimates 
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STS- 134 VNS Vertical Measurement Krror 
Tracking TCS reflector 1: bin hotel; includes B.E.T.-lo-sciuor adjustment 
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Figure 7.1-21. Estimated STORRM VNS Vertical Measurement Error; Tracking TCS Reflector 1 

Table 7.1-8 gives estimates of the mean differences and standard deviations of the bearing errors 
with and without the alignment of the BET to the data. 

Table 7.1-8 also provides estimates from the second-difference method. The estimates were 
obtained under the assumption the data are white noise [ref. 9]. The estimates under the 
assumption the data follow an AR(1) process that are computed by the method discussed in 
reference 15. Comparison between these two kinds of second-difference estimates shows the 
assumption that white noise yields underestimates, but the degree of underestimation is small 
compared to the difference between the BET-derived estimates and the corresponding estimates 
computed by second differences. The estimated errors shown in Figures 7.1-19 and 7.1-21 
suggest that residuals from a bias correction are closer to the small estimates of noise obtained by 
the second-difference method than the standard deviations given in Table 7.1-7. The horizontal 
and vertical measurements were fit as a function of time using loess with smoothing parameter 
(a) of 0.05 [ref. 16]. The estimated standard deviations of the noise from the loess residuals are 
given in Table 7.1-7. The estimated standard deviations obtained from loess residuals are closer 
to those obtained by the second-difference method than those derived from comparison to the 
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BET. This shows the second-difference method estimates the high-frequency noise well, but 
tends to eliminate the effect of slowly varying biases. 

The method chosen to estimate biases in the bearing data is less sophisticated than the range 
data. The bias is estimated as the mean of the differences between the BET-estimated 
measurements and the sensor measurements. When the bearing errors have a slowly time- 
varying bias, the bias corrupts the estimate of the standard deviation, making it larger. As 
explained previously, the team did not attempt to separate the error in the sensor from the error in 
the BET. This may increase the estimated error quoted. 

Interestingly, for Kalman filtering and optimal smoothing purposes, reference 9 recommended 
assuming the STORRM VNS bearing errors are uniformly distributed over a single pixel. Since 
each pixel corresponds to 0.0787 degree, the standard deviation of the bearing noise under this 
assumption is 0.0787/ ^fl2 = 0.0227 degree. The data in Table 7.1-8 show this value was closer 
to the estimates obtained by the BET-based method than the second-difference method that was 
used in the STORRM analysis contained in reference 9. 


Table 7.1-8. Mean Differences and Standard Deviations of Bearing Errors; STORRM VNS 
Tracking TCS Reflector 1 in Range Bin Hotel; N = 1138 


Mean diff. (deg) 

Std. Dev. (deg) 

Horizontal, not aligned 

0.644 

0.138 

Horizontal, aligned 

0.087 

0.070 

Horizontal, second difference (white) 


0.0019 

Horizontal, second difference, AR(1) 


0.0024 

Horizontal, from loess, a = .05 


0.0039 

Vertical, not aligned 

0.105 

0.055 

Vertical, aligned 

-0.149 

0.055 

Vertical, second difference (white) 


0.0018 

Vertical, second difference, AR(1) 


0.0021 

Vertical, from loess, a = .05 


0.0047 


Ninety-percent bootstrap confidence intervals on the standard deviations of the horizontal and 
vertical angle measurement noise, computed from the aligned values, are (0.067, 0.072) degree 
and (0.054, 0.057) degree, respectively. 

The sample autocorrelation coefficients at lag 1 for the estimated horizontal and vertical 
measurement errors are 0.9958 and 0.9970, respectively. 

7.1.3 STORRM Performance Summary 

Table 7.1-9 is a summary of the STORRM performance based on the analysis performed by the 
NESC team. 
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STORRM 

Bin 

Range 

Bounds 

(m) 

Est. Range 

Bias 

(m) 

Stdev of Range 

Error 

(m) 

Stdev of 
Horiz. 
Bearing 
Error 

(deg) 

Stdev of 

Vert. 

Bearing 

Error 

(deg) 

Hotel 

0 -30 

1.95-2.00 
(refl 1) 
2.01 -2.20 
(refl 14) 

0.031 (refl 1) 
0.044 (refl 14) 

0.070 
(refl 1) 
0.077 
(refl 14) 

0.055 
(refl 1) 
0.056 
(refl 14) 

Golf 

30 -81 

11.68 

11.676 (0.130 
omit 1) 

0.030 

0.043 

Foxtrot 

81 - 162 

2.15-2.34 

4.813 (0.169, 
omit 8) 

0.014 

0.178 

Echo 

162 -324 





Delta 

324 -648 

1.03-2.16 

4.988 (0.48, omit 
48) 

0.028 

0.087 

Charlie 

648 - 1300 

-1.69-0.21 

0.643 (omit 2) 

0.014 

0.032 

Bravo 

1300 -2590 





Alpha 

2590 -5760 






7.1.4 STORRM Pose-Based Range Estimates 

A review was conducted independent of the STORRM team to provide comments on the report, 
and to perform limited quantitative comparison of performance by comparing the VNS time of 
flight (TOF) range estimate to a pose-based range estimate for the portions of the trajectory 
where three retro-reflectors were available simultaneously. The pose-based estimates were 
compared to the VNS TOF range and the BET. 
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Comments on Reference 9 

The review of reference 9 did not lead to any significant observations. This reference is 
comprehensive, therefore a detailed examination of each section was not within the scope of this 
assessment. However, the sections that presented details of the VNS coordinate frames, centroid 
detection algorithms, and range data results were reviewed in detail. Since the report identifies 
the range estimates from the VNS in range bin Echo were of poor quality, it was not analyzed for 
the independent pose-based range estimates. 


Pose Estimate 

The second part of the review consisted of calculating a pose estimate for the STORRM VNS 
whenever three reflectors could be identified. The range estimate from the pose solution was 
compared to the VNS TOF solution. The range estimate from the pose solution is independent 
of the TOF solution, but in the same coordinate frame, which eliminates noise induced into the 
solution from coordinate frame transformations. With the published 1.918 meter bias of the 
VNS applied, the VNS data was in the range of 45 centimeters (cm) specified for the STORRM 
flight experiment. The exception to this is range bin Echo, which as noted in reference 9, had 
issues with the accuracy of the range estimate. The cause of this anomaly remains under 
investigation. 

A pose estimate works by comparing the geometry of at least three known locations on the target 
to the measurements by the sensor. If the Cartesian locations of at least three targets in the body 
frame are known, then three sides of a triangle can be constructed, as shown in Figure 7.1-22. 


Target 


Sensor 



Sensor measures bearing angles 
of spots (calculate three known 
angles of triangles, or 9 nn ) 


Target geometry 
is known (three 
known sides of 
triangles or r„„) 


The interior sides of the 
triangles are the unknown ranges 
to each spot (f?J 


Figure 7.1-22. Sensor to Target Geometry 


Additionally, the azimuth and elevation of each target spot measured by the sensor can be 
converted to three known angles. The three sides and three angles can be combined into three 
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equations with three u nkn owns by using the Law of Cosines. These equations can then be 
solved with a polynomial or iteratively to determine the individual ranges to each spot. 

R t 2 + R * - 2 R t R 2 cos( 6 12 ) — r ,2 = 0 

The procedure previously described for calculating a pose estimate of range to three targets was 
applied to Delta, Echo, Foxtrot, and Hotel range bins. As previously noted, the VNS data did not 
compare well in range bin Echo due to known problems with the STORRM VNS TOF range 
estimate in that bin. In all the other bins, the VNS met its experiment requirement of 45 cm 
when compared to the pose estimate. 

Comparison of the Pose Estimate to the STORRM VNS TOF Range and BET Range 
Estimates 

Figures 7.1-23 through 7.1-28 are plots of the pose estimate versus STORRM VNS TOF range 
and BET results from range bin Hotel. For this assessment, the three reflectors selected for the 
pose estimate were from the STORRM target assembly on the ISS. There are two figures per 
reflector. One shows the comparison of the pose range estimate to the raw (not corrected for 
bias) VNS TOF range estimate and the BET range estimate. The plots show the pose estimate is 
noisier at longer ranges in range bin Hotel. As the angles between the target reflectors 
decreased, the noise from the spot azimuth and elevations reduce. The gaps in the plots are 
because the pose estimate was not always possible since, in some cases, three target reflectors 
were not available. 

The BET data came from the reflector visibility tool, see Appendix I. This file contained the 
STORRM target assembly reflector centroids and the ranges for each ISS reflector as translated 
into the VNS sensor coordinate frame from the BET coordinate frame, which was published as 
the range between the orbiter and ISS respective CGs. In the process of translating the data from 
the orb iter- to-ISS coordinate from the VNS frame, additional noise may have been introduced 
into the BET. 
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STS-134VNS Raw and Pose Range Measurements Compared to BET-Estimated 
Measurements of Starboard STORRM Target Reflector in Range Bin Hotel 



Mission Elapsed Time (sec) x -j cf 


Figure 7.1-23. Overlay of TOF and Pose Range Estimates for Standoff Reflector 


STS-134VNS Pose Range and VNS Raw Range Errors 
from BET Tracking STORRM Target Standoff Reflector 



Figure 7.1-24. Estimated Range Errors of TOF and Pose Range Estimates for Standoff Reflector 
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STS-1 34 VNS Raw and Pose Range Measurements Compared to B.E.T.-Estimated 
MeasurementsofStarboardSTORRM Target Reflector in Range Bin Hotel 



Figure 7.1-25. Overlay of TOF and Pose Range Estimates for Starboard Reflector 


STS-1 34 VNS Pose Range and VNS Raw Range 
Errors from B.E.T. Tracking STORRM Target Starboard Reflector 
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Sensor-to-Reflector B.E.T. Range (m) 


Figure 7.1-26. Estimated Range Errors of TOF and Pose Range Estimates for Starboard Reflector 
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STS-1 34 VNS Raw and Pose Range Measurements Compared to B.E.T.-Estimated 
Measurements of Port STORRM Target Reflector in Range Bin Hotel 



Figure 7.1-27. Overlay ofTOF and Pose Range Estimates for Port Reflector 


STS-1 34 VNS Pose Range and VNS Raw Range 
Errors from B.E.T. Tracking STORRM Target Port Reflector 



Sensor-to-Reflector B.E.T. Range (m) 

Figure 7.1-28. Estimated Range Errors of TOF and Pose Range Estimates for Port Reflector 
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The STORRM VNS pose estimate and the VNS raw data are within 45 cm of the BET for range 
bin Hotel, as shown in Figures 7. 1-23 through 7. 1-28. These plots show the relative errors 
between the BET and the VNS raw (not corrected for bias) TOF and pose range estimates. 

While the BET is an estimate, and cannot be treated as absolute truth, for the purposes of this 
assessment, the VNS is considered to meet its experiment requirement of 45 cm for range bin 
Hotel. 

7.2 DragonEye 

7.2.1 Overview of DragonEye DTO 

The DragonEye DTO flew on STS-127 (July 2009) and STS-133 (February 2011). The STS-127 
unit was a prototype TigerEye design based on a terrestrial unit, which featured a low-grade 
(i.e., lower quality) detector array. During STS-127, the laser failed during the undocking 
portion of the DTO. The STS-133 flight employed an off-the-shelf DragonEye (which was 
derived from the TigerEye line). The unit was modified to bring the laser transmit optics closer 
to the receiving optics. The DragonEye unit for STS-133 had several improvements over the 
unit flown on STS-127. For both DTO flights, the DragonEye operated in the simplest of 
manner: during the test window, the unit was powered on and all data was recorded to a data 
recorder. At the end of the test window, the unit and the data recorder were powered off. Listed 
in Table 7.2-1 is the range bin setting for the DragonEye. Note the unit only had one set of 
gains. This was due in part to the simple DTO approach whereby no commanding or telemetry 
occurred in real-time. 


Table^2-L^ra^onE^eRan^eBin^ 


DragonEye 

Range Bounds 

Bin 


Standard Gain 

8 - 1000 m 

Setting 



There were three types of DragonEye data available to the assessment team. The first data set, 
delivered by SpaceX for the STS-127 flight, was a Matlab binary data file that contained the 
pixel location, intensity, and calculated range. The Matlab data was used in the NASA 
centroiding algorithm to generate centroids for this analysis. The STS-133 data consisted of 
SpaceX centroid data and DragonEye raw sequence data. The centroid data was generated by 
SpaceX’ s flight processing algorithms. The SpaceX centroid data was processed directly in the 
comparison to the BET. The raw sequence data was pre-processed so it could be fed into the 
centroiding algorithm. The DragonEye detector contains a 128 x 128 pixel array. Each “frame” 
contains 16384 pixels, and each pixel contains 20 intensity measurements (referred to as slices) 
and TOF index. The 20 slices and TOF index are processed to generate an intensity and range 
measurement for that pixel. This is performed for each pixel in the detector at every 
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measurement cycle (i.e., 688944 bytes for each frame). The processed raw sequence data was 
ready for the NASA centroiding algorithm. Reference 17 describes the process employed in 
converting the raw sequence data to usable format for the centroiding algorithm. 

7.2.2 NASA Centroiding Algorithm for DragonEye Data 

The centroiding algorithm used to process the DragonEye data is substantially different (and less 
sophisticated) than the STORRM VNS centroiding algorithm. These differences are largely due 
to the nature of the data and the amount of time available to build a customized processing 
algorithm. Specifically, the VNS has an angular resolution of 0.0787 degree/pixel, while the 
DragonEye has an angular resolution of 0.38 degree/pixel. Therefore, the angular resolution of 
the VNS is almost 5 times better than the DragonEye configuration flown on the STS-127 and 
STS- 133. This, coupled with a different choice for the focus of the receive optics, means a 
reflector spans multiple pixels for the entire rendezvous in the VNS data, while spanning only a 
single pixel for much of the DragonEye rendezvous data. Because a typical reflector return only 
illuminates a single pixel in the DragonEye data, the size and shape criteria developed for 
processing the VNS data were not applicable. Finally, by selecting a centroiding algorithm for 
the DragonEye data, reliably detected centroids at close range could not be determined where 
spurious returns dominate the LIDAR imagery. 

The DragonEye centroiding algorithm first applied a global threshold to the intensity image, 
keeping only pixels whose intensity is above a specified value. For the STS-127 and STS-133 
data, a reasonable value was found to be 2800 (maximum intensity = 4095). Second, a check on 
the range of the remaining pixels was performed and pixels with incorrect ranges removed. This 
was done by removing pixels with a range less than a specified value (usually around 3 meters). 
At this point, it was observed removing the edges of the image eliminated much of the remaining 
noise. Finally, a connected components analysis was used to extract contiguous blobs of pixels 
in the intensity image that passed the identified checks. These blobs were centroided using the 
same center-of-mass approach as for the STORRM VNS data. Many reflectors only contained a 
single pixel, making the center-of-mass computation unnecessary for much of the data. 

7.2.2. 1 DragonEye Range Measurement Performance Analysis 

Using the statistical methodology described in Section 6.5, this section presents the DragonEye 
performance in range and bearing sequentially. Figure 7.2-1 shows the DragonEye range 
measurements to TCS reflector 8 co-plotted with the corresponding sensor-to-reflector range 
computed from the BET. 
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■y. 


STS- 133 DragonKyc Range Measurements 
Compared to B JC.T .-Estimated Measurements 
Tracking TCS renector 8 



Mission-elapsed time (sec) 

Figure 7.2-1. STS- 133 DragonEye Range Measurements to Reflector 8 with BET-estimated Range 


After examining several potential OLS polynomial regression models, computing the 
autocorrelation of the residuals, and refitting using generalized least squares, the team used a 
quartic model without a linear term, based on a combination of engineering judgment and 
statistical lack-of-fit tests. Figure 7.2-2 shows the estimated error in the DragonEye range 
measurements to TCS reflector 8 with the estimated bias and 90 percent confidence bands. The 
bias departs from zero (at level 0.10) at all but the closest ranges. 
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STS- 133 Mean DragonEye Range Error vs. Range 
90% confidence hands: Tracking TCS reflector 8 
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Figure 7.2-2. STS-133 DragonEye Bias with 90 Percent Confidence Bands 


The noise was characterized by the standard deviation of the residuals. The estimates are shown 
in Table 7.2-2. The estimate from the second-difference method is a kn own underestimate 
because quantization error is a major contributor to the DragonEye range errors. 


Table 7.2-2. Estimated Standard Deviations of DragonEye Range Measurement Noise; Tracking 
TCS Reflector 8; N = 10510 


Method 

Estimated standard 
deviation (m) 

OLS regression of difference from BET, degree 1 

0.357 

GLS regression of difference from BET, degree 1 

0.357 

OLS regression of difference from BET, degree 4 ( (3i = 0) 

0.347 

GLS regression of difference from BET, degree 4 ((3i = 0) 

0.347 

Second-difference method (known underestimate) 

0.157 


A two-sided 90 percent confidence intervals on the standard deviation of the noise, while 
DragonEye tracked reflector 8 is (0.338, 0.358) meter, was derived from the GLS residuals. 
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Figure 7.2-3 shows estimates of the standard deviations of the range noise for ten range bins, 
with an estimated standard deviation function that is linear with range. The standard deviation of 
the range noise varies linearly as a function of range. Details on computing the standard 
deviation as a function of range are given in Appendix B. 

STS- 1 33 Dragonfly? Standard Deviations of Range Noise «. Range 
90% confidence intervals: Dragonfly? tracking TC'S reflector 8 
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Sciuor-to-reflcctor range (tn) 

Figure 7.2-3. Estimated Standard Deviation of the Range Noise as a Function of the Range 

DragonEye Range Measurement Performance while Tracking TCS Reflector 1 on STS-133 

Figure 7.2-4 shows the range measurements to TCS reflector 1 co-plotted with the corresponding 
sensor-to-reflector range computed from the BET. Figure 7.2-5 shows estimated DragonEye 
range errors and range bias while tracking reflector 1 on STS-133 and are shown are 90 percent 
confidence bands on the estimated bias. 
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Figure 7.2-4. STS- 133 DragonEye Range Measurements of Reflector 1 with BET-estimated Range 
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Figure 7.2-5. STS-133 DragonEye Bias with 95 percent Working-Hotelling and 90 percent 

Composite Bands 
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The DragonEye range errors are not normally distributed and have noticeable quantization error. 
Quantization is a result of the analog-to-digital resolution of the optical encoder. Of the 847 1 
residuals in the regression, 68 were larger than the others. The errors behave like a mixture of 
distributions. The 68 extreme residuals are approximately normally distributed with a mean of 
1.028 meter and standard deviation of 0.200 meter. If these 68 extremes are removed, then the 
estimated standard deviation of the noise was 0. 180 meter. 

As was the case in tracking reflector 8, the estimated standard deviation by the second-difference 
method resulted in an underestimate because of the presence of the quantization error. 

Table 7.2-3 shows a comparison of the estimated standard deviation of the noise in the 
DragonEye range measurements while tracking reflector 1 using multiple methods. The greatest 
reliance was placed on the GLS regression method. 

Table 7.2-3. Estimated Standard Deviations of DragonEye Range Measurement Noise; Tracking 
TCS Reflector 1; N = 8471 


Method 

Estimated standard 
deviation (m) 

OLS regression of difference from BET 

0.202 

GLS regression of difference from BET 

0.202 

OLS regression of difference from BET; omit 68 extremes 

0.180 

Quadratic OLS regression of range measurement vs. time 

2.996 

Loess with smoothing parameter = .20 

0.230 

Quadratic OLS regression of range vs. time; omit first 
1804 pts. 

0.459 

Quadratic OLS regression; last 3672 points 

0.190 

Second-difference method (known underestimate) 

0.050 


A 90 percent confidence interval on the standard deviation of the DragonEye range error while 
tracking TCS reflector 1 is (0.191, 0.216) meter, was derived from the GLS residuals. The noise 
decreases with range. 

7. 2.2.2 DragonEye Bearing Measurement Performance 

DragonEye Bearing Measurement Performance while Tracking TCS Reflector 8 on 
STS-133 

As mentioned in references 5 and 6, the TCS bearing and misalignment biases are u nkn own. 
They may be as large as 1 degree, though that is probably a worst case. A rotation was 
computed to attempt to align the BET and the DragonEye measurements [ref. 18]. It is not 
possible to separate the bias in the BET from the bias in the DragonEye bearing. The team 
reports the magnitude of the deviation between DragonEye and the BET, but did not ascribe it to 
the DragonEye bias. Estimates are reported with and without the alignment transformation, but 
plots are only shown for the aligned cases. 
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Figure 7.2-6 shows the horizontal angle measurements from DragonEye to TCS reflector 8, co- 
plotted with the corresponding estimates computed from the BET. The differences between the 
DragonEye horizontal measurements and the BET-estimated measurements are shown in 
Figure 7.2-7. 
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Figure 7.2-6. DragonEye Horizontal Measurements Compared to Altered BET Estimates 
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STS- 1 33 Horizontal Measurement Error 
Tracking TC'S reflector 8: includes B JC.T.-to-sensor adjustment 
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Figure 7.2-7. Estimated DragonEye Horizontal Measurement Error; Tracking TCS Reflector 8 

Figure 7.2-8 shows the vertical angle measurements from DragonEye to TCS reflector 8, co- 
plotted with the corresponding estimates computed from the BET. The differences between the 
DragonEye vertical measurements and the BET-estimated measurements are shown in 
Figure 7.2-9. 
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Figure 7.2-8. DragonEye Vertical Measurements Compared to Altered BET Estimates 
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STS- 133 Vertical Measurement Krror 
Tracking TCS reflector 8: includes B.K.T.-to-sensor adjustment 
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Figure 7.2-9. Estimated DragonEye Vertical Measurement Error; Tracking TCS Reflector 8 


Figure 7.2-9 shows during and following the orbiter’s MC4 the errors in the vertical angle 
measurement were larger than before MC4. The effects of the acceleration caused the increase 
in measurement error. RPOP records AV sensed by the orbiter’s IMU. On two consecutive 4- 
second RPOP navigation cycles, the AV were 0.272 and 0.267 meter/sec. DragonEye lost track 
of reflector 8 for about 20 second. After reacquisition of reflector 8, three consecutive AV of 
0.055, 0.055, and 0.038 meter/sec were recorded, and the vertical angle errors were unusually 
large at this time. The orbiter’s attitude and attitude rate changed during the maneuver, but its 
knowledge of the attitude was not degraded. During the same time DragonEye experienced 
vertical angle errors as large as 2 degrees, the TCS residuals in the RPOP Kalman filter were 
about 0. 1 degree. It can be concluded the DragonEye and TCS do not tolerate translational 
acceleration. 

In moving from the R-bar to the V-bar, the orb iter rotates with respect to the inertial frame at 
twice the orbital angular velocity. This rotation is designed to facilitate the commander keeping 
the target within the TCS field of view. The series of maneuvers from the R-bar to the V-bar is 
known as the Twice Orbital Rate V-bar Approach (TORVA). At the conclusion of the TORVA, 
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the rotation is nulled with respect to LVLH. Figure 7.2-9 shows the accuracy of vertical 
measurement estimation is impacted at the initiation of the TORVA. 

Figure 7.2-10 shows the magnitude of the AV while the translational maneuvers associated with 
the initiation of the TORVA occurred. The consequent increase in error in the vertical angle is 
seen in Figure 7.2-1 1. This was apparent by focusing solely on the period of time covered in 
Figure 7.2-10 and plotting the vertical error as a function of time. As can be seen in 
Figure 7.2-1 1, the vertical error grows approximately linearly with time. Note during this time, 
the twice-orbital rate rotation with respect to the inertial frame had not been initiated. Thus, the 
error is attributable to the translational AV, rather than the effect caused by rotation with respect 
to LVLH. 
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Figure 7.2-10. STS-133 Sensed IMU A V at the Beginning of the TORVA 
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STS- 133 Vertical Measurement Krror vs. Time 
Tracking TCS reflector 8: includes B.K.T.-to-scnsor adjustment 
After KP.M. at the beginning of the TORVA 
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Figure 7.2-11. STS-133 Vertical Measurement Error versus Time at the Beginning of the TORVA 


Table 7.2-4 gives estimates of the mean differences and standard deviations of the bearing errors 
with and without the alignment of the BET to the data. Recall each pixel represents 0.38 degree 
of angular measure. The statistics are given in pixels and degrees. 


Table 7.2-4. Mean Differences and Standard Deviations of Bearing Errors; DragonEye Tracking 

TCS Reflector 8; N = 10491 



Mean diff. (pix) 

Std. Dev. (pix) 

Mean diff. (deg) 

Std. Dev. ( 

Horizontal, not aligned 

4.143 

0.411 

1.574 

0.156 

Horizontal, aligned 

0.614 

0.390 

0.233 

0.148 

Vertical, not aligned 

0.450 

1.007 

0.171 

0.383 

Vertical, aligned 

0.306 

0.380 

0.116 

0.144 


Ninety-percent bootstrap confidence intervals on the standard deviations computed from the 
aligned values are (0.146, 0.150) and (0.142, 0.147) degrees for the horizontal and vertical 
measurements, respectively. The sampling error in the estimates of the standard deviations was 
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small, because the sample size is n = 10491. The sample autocorrelation coefficients at lag 1 for 
the estimated horizontal and vertical measurement errors are 0.9816 and 0.9864, respectively. 

DragonEye Bearing Measurement Performance while Tracking TCS Reflector 1 on 
STS-133 

This section evaluates the DragonEye performance in measuring the bearing to reflector 1 . 
Figure 7.2-12 shows the horizontal angle measurements from DragonEye co-plotted with the 
corresponding estimates computed from the BET. 


STS-133 DragonK>e Horizontal Measurements 
Compared to B JC.T .-Estimated Measurements 
Tracking TCS reflector I: includes B.E.T.-to-scnsor adjustment 
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Figure 7.2-12. DragonEye Horizontal Measurements Compared to Altered BET Estimates 


The quantization error evident in the range measurement data is even more obvious in the 
bearing data, because the DragonEye bearing measurements are quantized to a whole number of 
pixels. The focal plane array (FPA) resolution, FOV size, and focusing drives the system’s 
ability to do subpixel resolution, which creates quantized bearing measurements. The 
differences between the DragonEye horizontal measurements and the BET-estimated 
measurements are shown in Figure 7.2-13. 
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STS- 1 33 Horizontal Measurement Krror 
Tracking TCS reflector 1: includes B.K.T.-to-scnsor adjustment 
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Figure 7.2-13. Estimated DragonEye Horizontal Measurement Error; Tracking TCS Reflector 1 


Figure 7.2-14 shows the vertical angle measurements from DragonEye co-plotted with the 
corresponding estimates computed from the BET. 
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STS- 133 DragonEye V ertical Measurements 
Compared to B .E.T .-Estimated Measurements 
Tracking TC'S reflector 1: includes B.E.T.-to-sensor adjustment 
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Figure 7.2-14. DragonEye Vertical Measurements Compared to Altered BET Estimates 


The differences between the DragonEye vertical measurements and the BET-estimated 
measurements are shown in Figure 7.2-15. 
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STS- 133 Vertical Measurement Krror 
Tracking TCS reflector I: includes B.K.T.-to-sensor adjustment 
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Figure 7.2-15. Estimated DragonEye Vertical Measurement Error; Tracking TCS Reflector 1 


Figure 7.2-15 shows the error in the vertical angle grows in magnitude from acquisition of 
reflector 1 by DragonEye until the TORVA is completed. Completion of the TORVA is defined 
as the point the rotation of the orbiter with respect to LVLH as nulled. Figure 7.2-16 shows the 
in-plane relative motion during the time DragonEye tracked TCS reflector 1. Figure 7.2-17 
shows the vertical angle rate computed from the difference of consecutive BET-estimated angles 
divided by the delta-time. 
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Figure 7.2-16. STS-133 In-plane Relative Motion while DragonEye Tracked TCS Reflector 1 
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Figure 7.2-17. Estimated Vertical Angular Rate Computed from the BET 
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DragonEye does not tolerate rotation with respect to LVLH and performance was degraded by 
translational maneuvers, which was seen in tracking reflector 8. 

Table 7.2-5 gives estimates of the mean differences and standard deviations of the bearing errors 
with and without the alignment of the BET to the data. Each pixel represents 0.38 degree of 
angular measure. The statistics are given in pixels and degrees. 


Table 7.2-5. Mean Differences and Standard Deviations of Bearing Errors; DragonEye Tracking 

TCS Reflector 1 (N = 8471) 



Mean diff. (pix) 

Std. Dev. (pix) 

Mean diff. (deg) 

Std. Dev. ( 

Horizontal, not aligned 

3.978 

0.381 

1.512 

0.145 

Horizontal, aligned 

0.524 

0.401 

0.199 

0.152 

Vertical, not aligned 

0.484 

0.900 

0.184 

0.342 

Vertical, aligned 

0.289 

0.885 

0.110 

0.336 


Ninety-percent bootstrap confidence intervals on the standard deviations computed from the 
aligned values were (0.150, 0.155) degree and (0.330, 0.343) degree for the horizontal and 
vertical measurements, respectively. 

The autocorrelation coefficients at lag 1 for the horizontal and vertical measurement residuals are 
0.9825 and 0.9902, respectively. 

7.2.3 DragonEye Performance Summary 

Table 7.2-6 summarizes the DragonEye performance analysis performed by the NESC team. 
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DragonEye 

Bin 

Range 

Bounds 

(m) 

Est. Range 

Bias 

(m) 

Stdev of 
Range Error 
(m) 

Stdev of 

Horiz. 

Bearing 

Error 

(deg) 

Stdev of 

Vert. 

Bearing 

Error 

(deg) 

Standard 

Gain 

Setting 

25 - 126 
(refl 1) 

0.03-3.64 

0.202 

0.152 

0.336 

Standard 

Gain 

Setting 

196 - 585 
(refl 2) 

-1.11-0.90 

0.242 

0.102 

0.223 

Standard 

Gain 

Setting 

196 -259 
(refl 3) 

-0.77 - -0.09 

0.291 

0.135 

0.097 

Standard 

Gain 

Setting 

132 - 995 
(refl 8) 

-8.42-0.11 

0.347 

0.148 

0.144 


Since there quantitative specifications were not provided for the DTO, the team extracted the 
following DragonEye specifications from the Dragon vehicle requirements as a point of 
reference. Note these specifications were not applicable to the DTO, however, they do provide a 
quantitative reference to assess performance. 

• Range noise: ±15 cm (7 m < R < 50 m), ±100 cm (50 m < R 500 m), ±300 cm (500m < R 
< 1500 m) 

• Range bias: 10 cm + R*l% (7 m < R < 50 m), R*l% (50 m < R < 500 m), R*1.5 percent 
(500 m < R < 1500 m) 

• Bearing accuracy of ±0. 176 degrees 

All performance estimates were within the DragonEye vehicle specifications, with the exception 
of the vertical bearing specification that exceeded the specification during the tracking of 
reflector 1. 

From analysis of the raw data, the team found that pixel saturation, which degrades peak 
determination (a flat peak), impacts range computation. Identifying the peak intensity 
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determines the TOF, effectively the range estimate. Selecting the maximum intensity can lead to 
range errors starting at 34 cm. Applying a curve-fit or matching filter to the intensity trace may 
allow finer range resolution to select a proper peak (subslice precision). 

7.3 TriDAR Performance Analysis 

Neptec’s TriDAR flew on the Discovery on STS- 135 in July of 201 1. During the approach, a 
laptop on the orbiter was used to display TriDAR telemetry pages for the crew and to record 
data. The data used in this assessment was provided after NASA formally requested the data 
through the CSA. According to the conditions of the agreement with CSA, TriDAR raw data are 
not presented in this report, only differences from the BET. 

TriDAR is a scanning LIDAR and does not use retro-reflectors. It uses a structure model to 
compare the scanned laser returns to compute its relative state. This means the TriDAR can be 
used for relative navigation with an uncooperative target, provided there exists a model of the 
structure. In addition, in this section, performance is not based on individual retro-reflective 
targets, rather it is based on orbiter to ISS CG-to-CG relative location. 

The qualitative objectives of this TriDAR DTO were to fly the sensor, exercise the user 
interfaces, perfonn real-time pose estimation, and to demonstrate advancement of the TriDAR 
technology which was flown on two previous DTOs (STS-128 and STS- 131). The only 
quantitative objective was to acquire the ISS at a range greater than 1 km, which was 
significantly farther than had been demonstrated on previous DTOs. There were some general 
expectations of performance based on the ISS Relative Navigation System (RNS) specifications 
[ref. 19] for 3 sigma accuracy for range noise around 25 millimeters (mm) and bias of 50 mm, 
and bearing noise around 1.0 degrees. However, these specifications were not applied to this 
DTO. Table 7.3-1 shows the TriDAR range bins. 


Table 7.3-1. 

TriDAR Range Bins 

TriDAR Bin 

Range Bounds 
(m) 

6T VBAR 

1-21 

5T VBAR 

21-27 

4T VBAR 

27-52 

3T VBAR 

52-107 

2TR2V 

107-152 

1TRBAR 

152-228 

Acq 

228 - 1600 
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TriDAR operated in blob mode and 6-DOF pose. The blob mode calculates the range and 
bearing (azimuth and elevation) to the center of mass of the LIDAR scan returned point cloud. 
The 6-DOF pose mode processes the point cloud data and calculates the relative position and 
attitude of the SSACS frame with respect to the TriDAR frame, utilizing pre-loaded target CAD 
models. 

During approach and docking on STS- 135, the TriDAR acquired the ISS at a range of 2 km in 
blob mode. TriDAR entered pose mode before the RPM and then re-entered pose mode after the 
station was required post-RPM. In this analysis, the TriDAR data was analyzed in the blob mode 
pre-RPM and the 6-DOF pose mode post-RPM, omitting the data during the RPM and the pose 
data prior to the start of the maneuver. 

7.3.1 TriDAR - Performance Analysis 

Upon receipt of the data from Neptec, an initial review of the data was performed (detailed in 
Section 7.3.2). A BET for approach and docking to the ISS was generated for STS-135 and was 
used as a benchmark to evaluate the TriDAR performance. The BET pose is presented in orbiter 
to ISS CG. The TriDAR pose solution gives the SSACS location and attitude with respect to 
TriDAR frame. Therefore, accurate knowledge of the location of the TriDAR frame with respect 
to the orbiter to the ISS CG locations was needed to perform the coordinate translations. A 
diagram of the coordinate transformation process is shown in Figure 7.3-1. The orbiter attitude 
from the BET was needed to perform the comparison. The TriDAR data was corrected to 
account for the ODS pressurization error (described in the BET) and possible TriDAR mounting 
misalignments, which were derived empirically from the 6-DOF pose data. 

The following are steps used to perform the transformation: 

1 . Rotate TriDAR coordinate frame to the OBF. 

2. Adjust this rotation to account for ODS pressurization. 

3. Correct for TriDAR misalignment. 

4. Translate BET from ISS CG to SSACS origin. 

5. Use the orbiter attitude with respect to LVLH to rotate BET to OBF. 

6. Translate BET from orbiter CG to TriDAR focal point. 

Because the TriDAR system time was set once from a control laptop on-board the orbiter (it 
drifted over the course of the mission), it was necessary to find the time offset to align the 
TriDAR data to the BET. Finally, because the BET data points occurs every 4 seconds, 
interpolation was necessary to match the TriDAR telemetry data frequency of 5 Hz. 
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SSACS 



Figure 7.3-1. Coordinate Frame Translations from BET to TriDAR Pose 
7.3.2 Data Review and Anomalies 

The data delivered by Neptec contained TriDAR 3-DOF range and bearing (blob) and 6-DOF 
relative position and relative attitude (pose) states, which were computed on-board during 
approach and docking with the ISS. The data included time tags in coordinated universal time as 
recorded by a laptop, tracking state information, and data validity flags. Data with stale time- 
tags and invalid flag indicators were removed. Figure 7.3-2 shows the frequency of the stale and 
invalid data points. Most of the stale data occurred during blob mode at long range when the 
TriDAR was not updating the state as often as telemetry was being reported. 
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STS-135: TriDRR Data Valid/Stale 
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Figure 7.3-2. Frequency of Stale or Invalid Data in Early Phases of Trajectory (48893 to 50233 
seconds) (Impact of RPM and laser reset is visible) 


During the ISS approach, there were planned periodic resets of the TriDAR to test the 
reacquisition process. These resets occurred roughly every 10 minutes. The unit successfully 
demonstrated reacquisition after each reset. Periodic “reset” required stabilization time without 
user inputs (i.e., 30 seconds after reset command was issued to achieve “lock”). Figure 7.3-3 
shows these resets and how they correspond to the TriDAR tracking states. There was an 
unexpected laser reset that occurred during pose tracking mode 2T annotated in Figures 7.3-2 
and 7.3-3. The annotations in Figures 7.3-2 and 7.3-3 are left aligned with the event described. 
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STS-135: TriDRR Operational and Track State 
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Figure 7.3-3. Tracking State and Operational State (Note that the periodic resets (annotated) are 
clearly defined; tracking state during transition from REAR to YEAR seen after 52000 seconds, 

OpState cycling 1-3-5) 


7.3.3 TriDAR - Blob Mode 
7.3.3. 1 Range 

Range data acquisition occurred at a longer range (> 2 km) compared to the previous TriDAR 
DTOs due to laser hardware improvements. The blob range was compared against the BET 
(which starts at a range of 1600 meters) and the TriDAR estimated range error was plotted 
versus BET range (shown in Figure 7.3-4). Before blob-tracking lock started around 850 meters, 
sparse measurements of range and bearing were observed. The mean estimated range error was 
2.5 meters and the estimated range noise was 3.44 meters. After blob mode lock and until RPM, 
the TriDAR locked to one solar array instead of a whole ISS structure, leading to a bias in the 
range and bearing. 
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Figure 7.3-4. TriDAR Blob Mode Estimated Range Error 


133.2 Bearing 

The estimated bearing error in azimuth and elevation is shown in Figure 7.3-5. Note the majority 
of the bearing error is in the azimuth, consistent with the TriDAR tracking to one ISS solar array, 
rather than the entire station. At a range of 200 meters, the azimuthal bearing error is around 
15 degrees, which translates to approximately 50 meters (i.e., roughly the distance to the center 
of the solar arrays with respect to the center of the ISS). Before the blob mode filter lock, the 
standard deviation of the azimuth error was 1.09 degrees and the standard deviation of the 
elevation error was 0.26 degrees. 
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STS-135 TriDAR Blob Bearing Error Pre-RPM 



7.3.3.3 Summary of Statistics in Range and Bearing 

A summary of the estimated bias and noise for range and bearing is provided in Table 7.3-2. 
Because the estimates for range and bearing biases were calculated using a simple mean, the 
estimate of the noise is not a true measure of the residual error (as from a curve-fit), but the 
standard deviation from the mean. 


Table 7.3-2. Summary of TriDAR Blob Mode Statistics 



Before Blob Lock 

After Blob Lock 


Estimated Bias 

Noise 

Estimated Bias 

Noise 

Range (m) 

2.50 

3.44 

4.96 

6.23 

Azimuth (deg) 

-1.56 

1.09 

-7.99 

3.66 

Elevation (deg) 

0.80 

0.26 

0.93 

0.75 


NESC Request No.: TI- 11-00753 



NASA Engineering and Safety Center 
Technical Assessment Report 

Document #: 

NESC-RP- 

11-00753 

Version: 

1.0 

Title: 

Relative Navigation Rendezvous Sensor DTO 
Performance Evaluation 

Page #: 

87 of 103 


7.3.4 TriDAR - Pose Mode 

Once transitioning to the pose mode after RPM, the TriDAR used six different tracking strategies 
(Table 7.3-1) based on the relative range of the orbiter to the ISS. There was one tracking mode 
for the RBAR approach (IT), one tracking mode for the TORVA (2T), and four tracking modes 
for the final VBAR approach (3T through 6T). These tracking strategies modify the scanned 
area of the ISS. Therefore, the geometry of the structure model used for the pose estimation is 
based on the relative position of the orbiter with respect to the ISS. 

7.3.4.1 Pose Range 

The estimated pose range error as compared to the BET was used to assess the performance of 
TriDAR for the final approach and docking. Figure 7.3-6 shows the estimated range error for all 
the pose data post-RPM and demarks the six different tracking strategies. The average estimated 
range bias over the final approach was -0.92 meters and the range noise was 0. 145 meters. 
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It was observed a larger bias was introduced during tracking mode 2T (TORVA). This occurred 
when the TriDAR algorithm entered a local minima [ref. 20], possibly due to an unexpected a 
laser reset and power cycling. The average estimated range bias for tracking mode 2T was 
-0.92 meters with a range noise of 0. 14 meters. This phenomenon can be seen in Figure 7.3-3, 
where the TriDAR switches rapidly between tracking modes IT and 2T just after 52000 seconds 
(labeled TORVA on the figure). Figure 7.3-7 shows the estimated range error versus BET range 
separated into the z component of relative positions (which is in the range direction) and the 
root-sum-squared vertical and horizontal components of relative position (X-Y RSS) during the 
TORVA maneuver. Figure 7.3-7 indicates the bias is introduced in the z-direction and the X-Y 
components contribute to the noise. 


Tracking Mode 2T TORVA 



_-| i i i i i i i i i 

155 150 145 140 135 130 125 120 115 110 105 

BET range (m) 


-0.5 


E. 

2 -1 
C 

UJ 

N 


-1.5 

155 150 145 140 135 130 125 120 115 110 105 



BET range (m) 


Figure 7.3-7. 2T TORVA Estimated Relative Position Error 


There was a periodic variation in the range error found that can been seen in all of the range error 
plots and seen in Figures 7.3-8 and 7.3-9, tracking modes IT and 6T, respectively. It was 
determined this “ripple” effect was caused by periodic hardware health data gathering routines, 
which were “starving” the CPU every 5 seconds. Neptec is addressing this issue in their new 
TriDAR hardware. Since time-stamps are applied after the pose algorithm processing is 
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complete, if algorithm is running slow, the pose solutions can correspond to an earlier moment in 
time than reported. When the pose solution is fed to the filter with an incorrect time-stamp, the 
measurement is improperly incorporated into the filter algorithm and causes the “ripple” in the 
data. 
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Figure 7.3-9. 6T VBAR Estimated Range Error 

During VBAR acquisition, the estimated range noise improved to 0.042 meters as the TriDAR 
algorithms tightened in solution post TORVA (estimated range error is shown in Figure 7.3-10) 
In tracking mode 4T, the TriDAR perfonnance was at its best with the estimated range noise at 
0.021 meters. The contributions to the estimated range error and noise from the X-Y and Z 
components are shown in Figure 7.3-1 1. Note that a planned laser reset occurred at around the 
46-meter range and the pose algorithm recovered. However, there are outliers that can been 
found in Figure 7.3-1 1, which did not have a “data invalid” flag associated to them in the 
TriDAR telemetry and were used by the TriDAR pose algorithms and filter. 
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A summary of the estimated range bias and noise for all six tracking strategies is given in 
Table 7.3-3. 


Table 7.3-3. Summary of TriDAR 6-DOF Pose Statistics (range) 



Range Bin 

Estimated Bias (m) 

Noise (m) 


IT 

0.037 

0.207 
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2T 
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0.224 
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0.125 
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0.030 
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IT 

-0.006 

0.074 

a 

2T 

0.091 

0.085 

.0 

*-4— » 

3T 

0.144 

0.033 

o 

Pu 

4T 

0.153 

0.017 

X 

5T 

0.108 

0.032 


6T 

0.222 

0.093 

Ch 

IT 

0.049 

0.154 

.o 

4-» 

2T 

-0.001 

0.227 

o 

Cu 

3T 

-0.101 

0.084 

GO 

GO 

Pi 

>- 

X 

4T 

-0.205 

0.032 

5T 

-0.238 

0.040 

6T 

-0.207 

0.093 


IT 

-0.583 

0.097 

a 

2T 

-0.919 

0.143 

'O 

*-4-» 

3T 

-0.992 

0.046 

o 

Ph 

4T 

-0.931 

0.022 

N 

5T 

-0.893 

0.021 


6T 

-0.826 

0.044 

<D 

IT 

-0.577 

0.100 

RSS 

Rang 

2T 

-0.919 

0.143 

3T 

-0.998 

0.042 


NESC Request No.: TI- 11-00753 



NASA Engineering and Safety Center 
Technical Assessment Report 

Document #: 

NESC-RP- 

11-00753 

Version: 

1.0 

Title: 

Relative Navigation Rendezvous Sensor DTO 
Performance Evaluation 

Page #: 

93 of 103 



Range Bin 

Estimated Bias (m) 

Noise (m) 


4T 

-0.951 

0.021 

5T 

-0.923 

0.015 

6T 

-0.852 

0.048 


7.3.4.2 Pose Relative Attitude 

A similar analysis was performed for the relative attitude as for the range, separating the 
estimated relative attitude bias and noise for each tracking strategy. Table 7.3-4 summarizes the 
statistics for pose mode relative attitude. For brevity in this report, a plot of the relative attitude 
in pitch, yaw, and roll is shown in Figure 7.3-12 for the post-RPM approach and docking. The 
estimated relative attitude noise over the entire approach was 0.36 degrees in pitch, 0.5 1 degrees 
in yaw, and 0.63 in roll. 
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T able 7.3-4. Summary of TriDAR 6-DOF Pose Statistics (Relative Attitud e) 
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Noise (deg.) 
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0.048 


7.4 General Discussion of TriDAR, STORRM VNS, and DragonEye 

The range/bearing error plots for the TriDAR should not be directly compared to the STORRM 
VNS or DragonEye results. There are important nuances that make such a comparison difficult 
and likely lead to incorrect conclusions. 

The fundamental issue is the STORRM VNS and DragonEye DTOs focused on using the 
LIDARs for rendezvous with a “cooperative” target, while the TriDAR focused on rendezvous 
with a “non-cooperative” target. In the present context, the term “cooperative target” refers to a 
vehicle equipped with reflectors at kn own locations to aid the LIDAR in tracking specific points 
on the target vehicle. All of the range residual plots for the VNS and DragonEye were computed 
for specific reflector observations and the reflector identification was performed as part of the 
post-flight analysis (although this must be done on-board in real-time in an actual mission). In 
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contrast, during the “blob mode” used at long ranges, the TriDAR computes range/bearing to the 
center-of-mass of the observed 3-dimensional point cloud. Because the sensor is not tracking 
any known point on the vehicle, the best that can be done was to compare the result to the target 
CG. It should not be surprising to see a larger bias in the range error plots for the TriDAR 
because a specific point on the ISS for the sensor tracking is not being specified. 

For a significant portion of the STS- 135 approach, the TriDAR was only obtaining returns from 
one of the solar arrays, and was therefore computing the range/bearing to the center-of-mass of 
just this portion of the ISS (resulting in a bias relative to the ISS CG). Further, from 
measurement-to-measurement a different amount of the structure provided a sufficiently strong 
return to be included in this 3-D point cloud, leading to larger range noise than if one was 
viewing either the whole ISS or a specific point on the ISS. 

In contrast, the STORRM VNS and DragonEye DTOs were used to detect small reflectors 
(1-5 pixels in diameter, depending on range) over the entire rendezvous, thus avoiding the 
difficulties seen by the TriDAR in “blob mode.” STORRM VNS and DragonEye were tracking 
small objects (rather than observing only a portion of the object) mounted at kn own locations 
relative to the ISS CG. It should be stressed, however, that either the VNS or the DragonEye 
could be used in a “non-cooperative” mode rather than a “cooperative” mode. In addition, Soyuz 
solar arrays and windows produce strong returns on both flash LIDARS (STORRM and 
DragonEye). By adjusting the LIDAR sensor settings, the returns can be observed from the 
target (sometimes called “skin tracking”) as was done with the TriDAR. The primary difference 
now lies in how the data points are obtained - the TriDAR is a type of scanning LIDAR, while 
the VNS/DragonEye are flash LIDARs. 
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8.0 Findings, Observations, and NESC Recommendations 

8.1 STORRM VNS Findings, Observations, and NESC Recommendations 

F-l. The STORRM DTO met its objectives providing operational and performance data to 
continue system maturity. 

F-2. The STORRM calibration/correction and centroiding was performed post-flight. 

F-3. The performance of bin Echo (162 - 324 meters) exceeded the range specification. 

F-4. Biases were discovered in ground testing and from comparison to the BET. 

F-5. Discontinuities in range bin transitions (e.g., Alpha to Bravo were 90-meter jumps) and 
pose-based processing were discovered. This discontinuity was also observed in the 
pose-based processing. 

F-6. The use of second-differencing noise analysis led to underestimates of noise performance 
in some cases when time-based autocorrelation was present in the data. 

0-1. Marginal performance was observed in range bin Delta (324 - 648 meter) that may have 
been a precursor to the degradation that occurred in range bin Echo (162 - 324 meter). 

The following NESC recommendations are directed to the AR&D CoP. 

R-l. Conduct VNS processing development to mature the algorithms implemented within the 
sensor and navigation system to provide in-flight, on-board range, and bearing. 

(F-l, F-2) 

R-2. Determine the root cause of the performance issue of range bin Echo. (F-3, 0-1) 

R-3. Perform additional VNS preflight ground testing across all range bins to characterize 

system range performance. (F-3, F-4, F-5) 

R-4. Investigate the autocorrelation structure of time-series data before applying the second- 
difference method using polynomial or local regression methods. (F-6) 

• If the data appear uncorrelated, then the second-difference method may be 
appropriate. 

• If the data can be modeled as first-order autoregressive, then the second- 
difference method should be modified (see Appendix H). 

8.2 DragonEye Findings, Observations, and NESC Recommendations 

F-7. The DragonEye DTO met its qualitative objectives as performance specifications for the 
DTO unit were not established. 
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F-8. The DragonEye bearing measurements did not meet the Dragon vehicle specifications 
during tracking of reflector 1 planar. 

F-9. The intensity peak determination impacts range measurement for every pixel (i.e., as 
much as 34 cm error due to peak determination). Pixel saturation impacts ability to 
determine actual peak, and centroiding and feature identification. 

F-10. The DragonEye performance is adversely affected by host vehicle rotation with respect to 
the LVLH reference frame and when the host vehicle undergoes translational maneuvers. 

0-2. The FPA resolution, FOV size, and focusing drives the system’s ability to do subpixel 
resolution, which create quantized bearing measurements. 

0-3. There were significant design differences and maturity levels of the systems flown on 
STS-127 and STS-133. These include laser failure due to contamination and detector 
gain settings and performance (noisy pixels). 

The following NESC recommendations are directed to the AR&D CoP. 

R-5. Consider adjusting the camera design (e.g., the FOV and/or the focus) to achieve the 
identified specifications, or change. The specifications should reflect actual flight 
requirements. (F-8, 0-2) 

R-6. Apply a robust processing algorithm (e.g., curve-fitting) to find the peak intensity, which 
will drive the range measurement. (F-9) 

R-7. Notify users that DragonEye performance is adversely affected by host vehicle rotation 
with respect to the LVLH reference frame and when the host vehicle undergoes 
translational maneuvers. (F-10) 

8.3 TriDAR Findings, Observations, and NESC Recommendations 

F-ll. The TriDAR DTO met its qualitative objectives as performance specifications for the 
DTO unit were not established. 

F-12. There was a 1 -meter range bias that was unable to be reconciled over range from 
200 meters to ISS dock. 

F-13. In long-range blob mode, TriDAR identified a single ISS solar array rather than the entire 
ISS, which resulted in range errors of 20 meters and bearing errors of up to 50 meters in 
the y-axis. 

F-14. The data time tag was perfonned at the end of the processing resulting in asynchronous 
output of pose parameters due to CPU workload (e.g., health and status checking, 
infrared camera processing, etc.). 

F-15. Data quality issues included: stale measurement data that were flagged as valid, 
duplicated time-stamp data, and a lack of data quality flags to indicate validity. 
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0-4. Periodic “reset” required stabilization time without user inputs (i.e., roughly 30 seconds 
after reset command was issued to achieve “lock”). 

0-5. Unplanned laser reboot during 2T TORVA tracking due to laser power safety monitoring 
system trip resulted in a lower power setting (safe mode). Using high-power laser mode 
would be better or consider using eye-safe laser for manned operations. 

The following NESC recommendations are directed to the AR&D CoP. 

R-8. Determine the root cause of the 1 -meter range bias, which may involve reconciliation of 
CAD geometry, reference coordinate systems, and/or more extensive ground testing. 
(F-12) 

R-9. Direct TriDAR to output its scan window size for target returns in real-time as inputs to a 
navigation filter that can determine whether the scan is incomplete based on a priori 
target vehicle knowledge. (F-13) 

R-10. Apply time stamps real-time and add data quality flags to the TriDAR output. 

(F-14, F-15) 

8.4 Overall Findings, Observations, and NESC Recommendations 

F-16. When processing raw sequence or intensity and range data from Flash LIDAR sensors, 
data exclusion of erroneous range data based on the knowledge of FOV, range, and/or 
kn own bad/poor pixels is critical to measurement perfonnance. 

0-6. Soyuz solar arrays and windows produce strong returns on both flash L1DARS 
(STORRM and DragonEye). 

0-7. Flash LIDAR processing could encompass non-cooperative targets (i.e., non-centroid 
output processing). 

0-8. Numerous spurious reflections that were not LIDAR reflectors occurred for the 

DragonEye and STORRM VNS DTOs. By contrast, the TCS, which was a scanning 
LIDAR used during the SSP for proximity operations and docking, provided few 
spurious reflections throughout its lifetime. Spurious reflections make reflector 
identification and navigation difficult. 

The following NESC recommendations are directed to the AR&D CoP. 

R-ll. Perform algorithm development and testing to enable robust and reliable preprocessing of 
flash LIDAR imagery data. (F-l 6) 

R-12. Continue investments in STORRM VNS and DragonEye algorithm maturity for real-time 
flight applications (e.g., range calibration/correction and centroiding). (F-16) 
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9.0 Alternate Viewpoint 

There were no alternate viewpoints identified during the course of this assessment by the NESC 
team or the NRB quorum. 

10.0 Other Deliverables 

No unique hardware, software, or data packages, outside those contained in this report, were 
disseminated to other parties outside this assessment. 

11.0 Lessons Learned 

LL-l. For future LIDAR assessments, consideration should be given to resources required to 
obtain and archive the data sets. The DTO data required significant effort and resources. 
When conducting joint experiments, NASA should establish data-sharing requirements 
and agreements. 

12.0 Recommendations for NASA Standards and Specifications 

No recommendations for NASA standards and specifications were identified as a result of this 
assessment. 


13.0 Definition of Terms 


Corrective Actions Changes to design processes, work instructions, workmanship practices, 
training, inspections, tests, procedures, specifications, drawings, tools, 
equipment, facilities, resources, or material that result in preventing, 
minimizing, or limiting the potential for recurrence of a problem. 

Finding A relevant factual conclusion and/or issue that is within the assessment 

scope and that the team has rigorously based on data from their 
independent analyses, tests, inspections, and/or reviews of technical 
documentation. 


Lessons Learned Knowledge, understanding, or conclusive insight gained by experience 

that may benefit other current or future NASA programs and projects. The 
experience may be positive, as in a successful test or mission, or negative, 
as in a mishap or failure. 

Observation A noteworthy fact, issue, and/or risk, which may not be directly within the 

assessment scope, but could generate a separate issue or concern if not 
addressed. Alternatively, an observation can be a positive 
acknowledgement of a Center/Program/Project/Organization’s operational 
structure, tools, and/or support provided 
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The subject of the independent technical assessment. 

The event(s) that occurred, including any condition(s) that existed 
immediately before the undesired outcome, directly resulted in its 
occurrence and, if eliminated or modified, would have prevented the 
undesired outcome. 

A proposed measurable stakeholder action directly supported by specific 
Finding(s) and/or Observation(s) that will correct or mitigate an identified 
issue or risk. 

One of multiple factors (events, conditions, or organizational factors) that 
contributed to or created the proximate cause and subsequent undesired 
outcome and, if eliminated or modified, would have prevented the 
undesired outcome. Typically, multiple root causes contribute to an 
undesired outcome. 

Supporting Narrative A paragraph, or section, in an NESC final report that provides the detailed 
explanation of a succinctly worded finding or observation. For example, 
the logical deduction that led to a finding or observation; descriptions of 
assumptions, exceptions, clarifications, and boundary conditions. Avoid 
squeezing all of this information into a finding or observation 


14.0 Acronyms List 

AA 

Associate Administrator 

AFT 

Aft pointing 

AR 

Auto-regressive 

AR&D 

Autonomous Rendezvous and Docking 

BAH 

Booz Allen Hamilton 

BET 

Best Estimated Trajectory 

CG 

Center of Gravity 

cm 

centimeters 

CoP 

Community of Practice 

CSA 

Canadian Space Agency 

DC 

Docking Camera 

deg/sec 

degree/second 

DoF 

Degree-of-Freedom 

DRU 

Data Recording Unit 

DTO 

Development Test Objective 

FD 

Flight Day 

FGB 

Functional Cargo Block (RSA module) 

FOR 

Field of Regard 


Problem 
Proximate Cause 

Recommendation 
Root Cause 
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FOV 

Field-of-View 

FPA 

Focal Plane Array 

FWD 

Forward pointing 

GLS 

Generalized Least Squares 

GN&C 

Guidance, Navigation, and Control 

GSFC 

Goddard Space Flight Center 

GUI 

Graphical User Interface 

HEOMD 

Human Exploration and Operations Mission Directorate 

Hz 

Hertz Cycles per Second 

IMU 

Inertial Measurement Unit 

IR 

Infrared 

JEM 

JAXA Experiment Module 

JSC 

Johnson Space Center 

km 

kilometers 

LaRC 

Langley Research Center 

LIDAR 

Light Detection and Ranging 

LVLH 

local vertical, local horizontal 

m 

meter 

mm 

millimeters 

MPCV 

Multi-Purpose Crew Vehicle 

NESC 

NASA Engineering and Safety Center 

NRB 

NESC Review Board 

OBF 

Orbiter Body Frame 

ODRC 

Orbiter data reduction complex 

ODS 

Orbiter Docking Station 

OLS 

Ordinary Least Squares 

OSC 

Orbiter Structure Coordinate 

PI 

Principal Investigator 

PMA 

Pressurized Mating Adaptor 

RelNav 

Relative Navigation 

RNS 

Relative Navigation System 

RPOP 

Rendezvous and Proximity Operations Program 

SEA 

Sensor Enclosure Assembly 

SM 

Service Module (RSA Module) 

SME 

Subject Matter Expert 

SSACS 

Space Station Analysis Coordinate System 

SSP 

Space Shuttle Program 

STS 

Space Transportation System 

TCS 

Trajectory Control Sensor 

TOF 

Time of Flight 

TORVA 

Twice Orbital Rate V-bar Approach 
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TriDAR Triangulation and LIDAR 

TRL Technology Readiness Level 

VNS Vision Navigation Sensor 
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